devopsy.kinsta.cloud

Author: Bhanu Singh

  • Legacy Systems: Should They Stay or Should They Go?

    Legacy Systems: Should They Stay or Should They Go?

    Digital transformation requires new technologies. But does that mean legacy systems should be replaced?

    Survey after survey has shown that IT and business leaders have had to drastically rethink priorities, strategies and spending in the wake of the global pandemic. A survey our company conducted last October showed that technology spending hasn’t taken a terrible hit—in fact, 60% of IT leaders reported they had either significantly or moderately increased their IT budgets in the last six months. Other research from 2020 mirrors those findings.

    Yet, there are significant challenges. Three top ones reported in the OpsRamp survey include business and IT alignment, available technology solutions and economic uncertainty. Executives want to move forward, yet they need to be cautious with budgets and develop strategies that can flex with the unpredictability of 2021.

    And this is how we arrive at the topic of legacy systems. As part of the reprioritization effort, IT leaders are wise to take a hard look at their technology portfolio and determine which systems should go and which should stay—factoring in economics, business needs, customer demands and change management requirements. Progressive IT leaders know that if they don’t adopt the right technology at the right time, continued business success (and even survival) is at risk.

    Defining Legacy Systems

    We tend to associate a legacy system with a technology that’s been around a long time, yet it could also be a three-year-old system. Legacy systems can be a burden when the UI and the UX look dated, are slow or difficult to upgrade and integrate with other applications, and costly to maintain. Many legacy systems were built for on-premises environments and don’t migrate easily to the cloud. Finally, because some systems are built on languages and technologies that are 10 to 20 years old, IT organizations struggle to find people with the requisite skills to maintain them. These are all red flags.

    On the other hand, there are valid reasons why enterprises maintain legacy systems:

    • The technology technically isn’t broken and satisfies a core business process, industry requirement or customer need.
    • The system integrates with many other applications and core processes, making it difficult to replace.
    • Longstanding vendor relationships make it politically or culturally challenging to abandon.
    • Influential stakeholders love it, despite any negative impact on customers and revenues.

    How to Evaluate a Change

    Even if a system still currently meets your business need, it’s important to understand what your competition is doing and what your customers are asking for to ensure you’ve got a future-proof strategy. Consider Slack, Amazon, Google and Netflix: These innovators have redefined what experiences software can provide and thereby have changed expectations for all users regardless of the sector.

    In IT operations, if a customer is receiving fewer IT alerts due to filtering and correlation capabilities, that’s a good thing. But does the software proactively manage those alerts, filtering out the high-priority items to the right person and suggesting the best next steps for a quick resolution before users are affected?  From the executive and business view, you need to demonstrate efficiencies and cost savings from moving to a new solution as well as revenue-generating impacts.

    Mitigating Snafus With Legacy System Replacements

    When you decide it’s time to make the big move and replace a major system, consider the top factors that can ruin these projects or needlessly delay the time to value:

    • Gaps in knowledge and mindset. Look at what happened when every CIO got a mandate from the CEO to move operations to the cloud. Understanding how the cloud works and how to manage cloud-based assets was a struggle, so most early cloud migrations were simply lift-and-shift. That led to a common scenario in which not only did companies not achieve the savings they expected, but also the environment became even more expensive to support than their on-premises infrastructure.
    • New skills. In lockstep with the gap in strategic understanding of new technologies is the fact that it’s difficult to acquire or build the associated new skills fast enough to keep pace with changes in the market. This is why still today, more than 10 years after the advent of public cloud, there are significant talent gaps in the areas of cloud architecture, cloud optimization and cloud-native development.
    • Cultural changes. People don’t like change, especially if they’re not unhappy with the status quo. IT leaders are improving here, but there’s still ample work in helping convey the reason for change and the benefits to designing a frictionless path to adoption.

    The 3-step Plan to Modernization

    There’s nothing especially new about how to avoid a marred enterprise software implementation, except for the fact that tolerance levels are at an all-time low. Executives don’t want to see one-year rollouts and two-year time-to-value cycles anymore. Here’s how to simplify and speed up the benefits when moving to new platforms:

    1. Start with a small, powerful punch. You can gain momentum with a small project that doesn’t commit the organization to a gargantuan, tangled overhaul. Pick a project that’s not going to stress out your team and has a specific, predictable ROI. Truth is, people love innovation and want to be part of the “next new thing” as long as it doesn’t require too much pain all at once. Make sure to socialize the outcomes across the organization and explain how you did it.
    2. Make the leaders your champions. If your executive team doesn’t feel motivated to promote a new system internally, you might want to investigate whether it’s the right solution. Be sure to get that buy-in before you sign any vendor contract. Communication style matters. Who on the executive team can really create excitement around the future and effectively demonstrate the need for change? You want a salesperson personality for this job.
    3. Foster an evangelist committee. In the trenches, you also need a team of people who understand how to use the new solution and are eager to share their tips. I’ve seen leaders take on a full internal campaign prior to a new system being launched, from rebranding and relabeling an enterprise software package, developing an internal PR strategy around it and creating really effective tools such as short videos to help users get up to speed.

    Balancing Continual Innovation and Lower Complexity

    A legacy system overhaul is not for the faint of heart. You must carefully consider whether your organization is ready to take on transformational change or if an incremental change, such as customization or a new integration to your existing platform, is enough for now. But if incremental change is just going to help you survive another day, it can make the necessary change you really need much more painful later on.

  • Engineering Productivity Lessons From COVID-19

    Engineering Productivity Lessons From COVID-19

    The world became a different place post-COVID-19 and how we work, communicate and collaborate has been redefined, possibly forever. Our company has a distributed team across the U.S. and India for development and operations. When “shelter in place” hit the entire world, we were not sure how we’d be able to execute our projects, customer commitments and day-to-day operations. Most of our engineers rarely worked from home, and in India, some even left their homes to be with extended families and parents during the lockdown.

    In first-ever crises like the pandemic, leadership teams have to rethink everything to ensure safety for employees while keeping them engaged, motivated and performing during highly disruptive and stressful times.

    Maintaining During COVID-19

    Here are five strategies and tactics that we applied and iterated to help our teams focus and execute during very unusual and difficult times.

    1. Embrace the change.
    2. Listen to employees.
    3. Empower staff at every level.
    4. Communicate, communicate and communicate.
    5. Facilitate outcome-driven engagement, development and execution.

    Embrace the Change

    It was important for the leadership team to be intentional and systematic in accepting the change and establishing proper guidelines, tools and processes to transition quickly to a work-from-home environment. Selecting the same tool(s) for everyone established consistent, exclusive communications to help mitigate isolation. Some employees needed extra guidance in setting the workplace, schedule and systems to be effective working remotely.

    Listening and Sharing

    Employee health and safety was the top priority when lockdowns began, and it started with active listening to employees. Many employees needed to discuss the latest news on the pandemic or express concerns about loved ones who had contracted the virus. We created different levels of informal and formal check-in sessions in the early days of “shelter in place” to discuss anything, from family challenges including child care and remote school to news and market updates. Having open conversations about shared stress during the onset of the pandemic proved valuable in creating mutual support that we all needed to carry on with our daily obligations at work and at home.

    Empowerment at Each Level

    As we enter this next phase of COVID-19, with many of us fully engaged in some form of “new normal,” we have focused on aligning producers and consumers of deliverables to ensure they are fully engaged and understand the dependencies. This entails asking for individual accountability on deliverables and also encouraging people to work on aspirational goals.

    We were already practicing objectives and key results (OKRs), but COVID-19 made it clear we needed to push harder on further refining OKRs and helping individuals prioritize and rationalize their goals as they progress. The goal-setting through OKRs has been super helpful in bringing our remote employees together on the same page with shared initiatives and missions.

    Communicate and Overcommunicate

    Frequent communications and collaboration across teams was a priority, allowing managers to ensure a clear line of sight into deliverables, priorities, trade-offs, logistical challenges and dependencies. The overarching goal was to keep people productive and avoid unnecessary delays in projects.  While it was important in the early stages of the pandemic to overcommunicate across teams, in time, installing the right processes, tools and objectives drove engagement.

    Outcome-driven Development and Execution

    We use KPIs to measure outcomes on a daily and weekly basis. The outcome could be as simple as fortification of certain buggy areas of the product or security testing and fixing or automating defects. Outcomes can also be future looking cool experimentations that drive product ideas and generate compelling customer value. But the key is to make sure the team understands what success looks like throughout the journey of driving the execution of the outcome to the finish line. A simple example could be to review all security CVEs for open source libraries used in the product and ensure they are updated with the latest security patches.

    We were surprised and delighted to see employees taking matters into their own hands by doing more than they were asked, especially in cases where they could bring their creativity to the table. As leaders, we sometimes undermine the power of an empowered team and the magic it brings to add measurable customer and business value.

    The last few months have been taxing on everyone around the world; we do not know what things will look like in six or 12 months. Yet, we’ve also had the opportunity to learn some invaluable lessons about leadership, management and team culture.  Moving forward past this pandemic’s inevitable conclusion, individual empowerment with outcome-based objectives will be the new recipe for engineering success. We’ve seen first hand how human creativity can get a boost when employees are motivated to share their stories, display their ideas no matter how rough they are and work together toward clear goals and a common shared vision.

  • Edge Computing and ITOps: Opportunities and Challenges

    Edge Computing and ITOps: Opportunities and Challenges

    Edge computing is growing quickly, but IT operations need to evolve as well to effectively monitor and manage new devices, sensors, applications and data. Growing enterprise edge ecosystems should integrate and become part of centrally managed ITOM systems.

    Edge computing is hard to define and is running high on the hype scale. But, research and surveys continue to indicate that this trend of processing data where it’s collected for better latency, cost savings and real-time analysis is an innovation with legs.

    There will be 75 billion IoT devices by 2025, according to Statista. According to Spiceworks’ 2019 State of IT report, 32% of large enterprises with more than 5,000 employees are using edge computing, and an additional 33% plan to adopt it by 2020. Tied to the growth of edge computing is the advent of 5G wireless: 51 operators globally will start 5G services by 2020, according to Deloitte Global research from 2019.

    The major cloud companies are also investing in the edge. The AWS Local Zones service allows single-digit latency connecting to computing resources in a metro environment, while Microsoft offers the Azure Stack Edge appliance and Google Cloud IoT is a “complete set of tools to connect, process, store and analyze data both at the edge and in the cloud.”

    It’s safe to say that edge computing is becoming mainstream, and CIOs and their ITOps leaders should plan appropriately for it in 2020 and beyond.

    Benefits of the Edge for ITOps

    We’ve read plenty about the business benefits from edge computing: oil rig operators need to see critical sensor data immediately to prevent a disaster, marketers want to push instant coupons to shoppers while in the store, video security monitoring can catch a thief in the act and medical device alerts that ensure patient safety. Edge computing may save IT money on cloud and network bandwidth costs as data volumes keep exploding and the need to store every data point becomes harder to justify.

    There are also implications for IT management and operations. Local processing of high volume data could provide faster insights to manage local devices and maintain high-quality business services when seconds make a difference—such as in the event of a critical server performance issue threatening the e-commerce site.

    Today, ITOps teams are inundated with data from thousands of on-premise and cloud infrastructure components and an increasingly distributed device footprint. The truth is, only an estimated 1% of monitoring data is useful, meaning that it provides indications of behavior anomaly or predictions about forthcoming change events. With edge monitoring, we can potentially program edge-based systems to process and send only that small sliver of actionable data to the central IT operations management system (ITOM), rather than transmitting terabytes of irrelevant data daily to the cloud or an on-premise server where it consumes storage and compute power. 

    The job of filtering out the highly-contextual data on the edge, where business occurs, can support real-time decisions for successfully running ITOps at speed and scale—regardless of what combination of on-premise, public cloud or private cloud infrastructure is in place. At the same time, ITOps will need to be a leader when it comes to minimizing the risk of edge technology from a performance, security and privacy perspective. However, as detailed below, we are in the early stages of determining how to make this work in practice.

    ITOps Realities for Edge Computing 

    Edge-Specific Security Needs Are Still Unknown: Edge devices are often small and infrequently designed with security in mind. More than 70% of edge devices don’t mandate authentication for third-party APIs, and more than 60% don’t encrypt data natively. So the attack surface in IoT and edge environments is now larger, and less secure. This is particularly worrisome when considering edge devices that collect personally identifiable information such as email, phone numbers, health data or financial formation such as credit card data. ITOps will need to work closely with security and legal teams to map out the company-specific risk, governance and compliance requirements around managing edge data.

    Edge Monitoring Tools Are Immature: Companies need platforms that can instantly monitor and analyze edge-generated data. In the connectivity of tomorrow, billions of connected devices will be communicating machine-to-machine, and the addition or subtraction of connected devices will be possible at an unprecedented scale. In this environment, the ability to manage large volumes of connected devices and the information being exchanged between them will be critical. 5G acts as the unifying technology, bringing flow of information and the density of scale. We will see an influx of innovation in edge monitoring in the coming years.

    New Environments Call for New Rules: As organizations move more data and application assets to edge computing environments, IT will need to devise new policies and thresholds for central processing and alerting of all this data. Applying AI-based automation is essential here, as manual efforts will have zero chance of keeping up with the volume of data filtering, analysis and response. We are entering the age of nano satellites, vis-à-vis SpaceX and OneWeb. These edge devices will transform the future of agriculture, energy, mining, transportation and finance due to their capabilities for sending insightful data in real-time to customers, wherever they are at any moment. ITOps will have its work cut out to understand and properly manage this evolving edge infrastructure.

    DevOps Processes Will Become Even More Paramount: If you haven’t already realized that DevOps is taking over software development and IT management, just wait for when edge goes mainstream. There will be no other way to manage change and deployments of edge technology without the agile, continuous integration and continuous delivery methodology of DevOps. It will be imperative for ITOps to adopt DevOps practices and tools to manage, monitor and deploy edge resources.

     

     

     

    ITOps is at a crossroads, determining how much of the past is still relative and how much they will need to change to adapt to a distributed, hybrid cloud world that will soon include edge as a fundamental pillar of their digital strategy. Security, machine intelligence and DevOps will be crucial areas of expertise for ITOps teams looking to help drive better business value and customer experiences from the edge.

    — Bhanu Singh

  • How Systemic Approaches to ITOps Impact the Business Culture

    How Systemic Approaches to ITOps Impact the Business Culture

    We are at an inflection point in the rise of big data. Enterprise IT infrastructure is producing data at a scale like the world has never seen before. IT operations (ITOps) management has accelerated past the notion of data storage and processing (vis a vis the Hadoop model) and into the world of analytics-as-a-service. We now need systems to drive insights and manage change on-demand. In fact, we need a new operating model for IT in the face of these new effects.

    Can a New System Save Us?

    The key to transforming into a responsive, data-driven enterprise—while avoiding a flood of unnecessary processes and over-staffing—is to adapt automation to intelligently consolidate information flows. At the same time, the dynamic nature of today’s IT environment requires IT teams to become literate in a completely new approach to technology, one that spans all functions of the business. Consequently, risks that previously stayed inside organizational units now leak across domains, influencing decision-making for the entire company. These factors will try to pull the organization in multiple directions at once; only a unified data management system with the intelligence to extract operational insights can drive positive business change.

    Culture Has Got to Change, Too

    Agile processes such as DevOps and AIOps offer mechanisms to control the flow of data and turn it into actionable results. For example, continuous application development and deployment gathers data about functional use, user sentiment, efficiency and various other metrics to inform the overall process and improve the product. Such rapid dev cycles let organizations identify and fix issues more quickly and with fewer resources, ultimately delivering greater business value.

    The resulting streamlined collaboration among IT, development and customer teams improves capabilities but also introduces dependencies that must be managed effectively. It changes the culture to adapt to the system, and thereby, the system of service delivery will fundamentally affect how organizations are structured, built and run. Eventually, there will be increased digitization of every human process, and that means human impact slowly will be replaced by AI impact. This doesn’t mean that machines are coming to take our jobs; instead, it means the modern enterprise IT organization must address a new future. How will it look?

    Retraining and Reskilling Is a Fact of Life: In a world of dynamic IT, requirements are no longer static, and teams need to be prepared. Organizational units must be able to work in different ways with new technology, with new skills that include robotic emotional intelligence, cloud-native literacy and infinite adaptability.

    System operators become stewards of the business because system operations directly impact business outcomes. It won’t be enough for workers to merely deploy, configure and manage IT while isolated from the goals of the company, because those goals are inextricably linked to how objectives are achieved through innovation. Employees will need to develop new skills for functioning in a dynamic system. Training staff in agile methodologies, service-oriented software design and six sigma-style approaches for process improvement will become critical for success.

    Automation Is Everything: Automation by policy won’t be enough; even runbooks will become obsolete. Data and results are always changing, so simple automation will not drive meaningful change. AI can be used to drive automation policies and frameworks, with the potential to reduce errors and integrate disparate systems, which previously required multiple points of oversight.

    The trend toward using AI for data understanding is driving intelligent services into other parts of the business, as well. Rather than just for correlating system and operational data, AI is being adapted to business process optimization. Moving forward, IT operations can begin to run themselves, led by deep data analytics. Alerting and response processes automatically will use their own feedback to update the system’s intelligence, identify emergent trends and take actions to deliver improved operational results.

    Risk Becomes Universal: Increasing connectivity and generalized frameworks (such as DevOps and site reliability engineering) mean that risk is no longer siloed to one department or function. It’s spread everywhere, and cascading effects become commonplace.

    In general, successful engineering outcomes rely on coordinated processes and policies that dictate operational needs, even after deployment. Introducing connectivity between formerly disparate systems can create instability that potentially affects operations in other unintended areas. Thus, broad risks arise if different systems can’t interoperate once unified into a single IT ecosystem. Addressing requirements under a generalized framework helps prevent gaps between application implementations.

    The System Drags the Company Forward: Insights-driven action will transform digital business well in advance of executive priorities. Change will happen before anyone is ready. Indeed, with so much complexity, so many tools and processes and so many competing business demands, IT changes now impact culture in entirely new ways.

    Business impacts will come from system intelligence rather than manual processes. Efficiency in operations management is derived from the entire system, including the people themselves. What was previously the IT culture is rapidly being replaced with a combination of self-adapting processes and workers who focus more on value than on plumbing.

    Transforming the enterprise from silos into integrated platforms will propel the business forward. The resulting system will also drive cultural change at an accelerated rate, perhaps faster than the worker community is prepared to accept, but informed by real data instead of executive intuition.

    — Bhanu Singh

  • What Can Automation Learn From DevOps?

    What Can Automation Learn From DevOps?

    A recent Capgemini survey revealed that while enterprise-scale automation is still in its infancy, IT automation projects are moving along. IT is starting to view automation less tactically and more strategically.

    IT leads automation implementation (respondents were asked to select all that apply: “In which of the following functions has your organization implemented automation initiatives?”). Capgemini Research Institute, Automation Use Case Survey; July 2018, N=705 organizations that are experimenting with or implementing automation initiatives.

    The Capgemini survey also showed that IT automation can be responsible for several quick wins, including self-healing, event correlation, diagnostics, application releases, cybersecurity monitoring and storage and server management tasks. These projects not only lead to massive IT cost savings but, more importantly, to an increase in reliability and responsiveness to customer demands and business services. That would indicate that while automation is a great solution for manual work, it’s also a part of a high-level, strategic IT plan to innovate the business.

    As DevOps practices such as agile methodology, continuous deployment and optimization start to take hold within the modern enterprise, it stands to question: Can automation be agile as well? This is the promise of artificial intelligence (AI) for IT operations, or AIOps, but if that’s not a possibility for your IT organization today, it’s important to make sure that your automation practices are continuously optimized to fit the task. Setting and forgetting was a practice of the server era, and in a world of on-demand infrastructure, automation ought to be continuously optimized and evaluated for maximum benefit.

    The New Expectations of Automation

    IT automation projects can have serious ramifications if anything goes wrong because when the machines execute a policy, they do it in a big way. This is perhaps the chief argument as to why it’s critical that progressive steps are used to define and evaluate both the process being automated and the automation itself—they mitigate the seriousness of any issue that can arise. This is why it’s important to consider the following:

    1. Is this a good process, and is it worth automating?
    2. How often does this process happen?
    3. When it happens, how much time does it take?
    4. Is there a human element that can’t be replaced by automation?

    Let’s break down these steps and see how it can provide the basis for an iterative approach to automation:

    Is this a Good Process?

    This may seem like a rudimentary question, but in fact, processes and policies are often set and forgotten—even as things change dramatically. Proper continuous optimization or agile automation development will force an IT team to revisit existing policies and identify if it’s still right for the business service goals.

    Some processes are delicate and automation may threaten their integrity, whereas others are high-level and automation neglects the routine tasks that underlie the eventual results. A good automation engineer understands what tasks are best for automation and sets policies accordingly.

    How Often Does this Process Happen?

    Patching, updating, load balancing or orchestration can follow an on-demand or time-series schedule. As workloads become more ephemeral moving to serverless cloud-native infrastructure, this process’s schedules will change as well. An automation schedule ought to be continuously adapted to the workload need, customer demand and infrastructure form. As the business continues the march toward digital transformation, the nature and schedule of particular work may become more dynamic.

    When It Happens, How Much Time It Takes?

    This also depends on the underlying infrastructure. Some legacy systems require updates that may take hours and some orchestration of workloads will be continuous. Automation must be tested to be efficient and effective on the schedule and frequency of the manual task.

    Is There a Human Element that’s Irreplaceable?

    As much as you may want to, it’s difficult for automation to shift left (to more experienced tasks and teams) without the help of AI or machine learning. Many times, there is a human element involved in deriving insights, creating new workflows, program management or architecture that take place. When building an iterative automation practice, make sure you identify where human interaction must occur to evaluate and optimize. In our lifetime, technology has advanced at lightning speed, with robots now completing jobs that were once held by people. However, there are times when a machine just cannot deliver the same quality a human can.

    Automation for All

    Automation is perhaps one of the most defining signatures of the future of IT operations management. It relieves teams of routine work and helps improve overall efficiency, all while driving quick wins that turn an IT team into heroes. But don’t let automation be the end goal. Instead, consider it a tool that can drive action from data. And until AI is an everyday option, it’s inherent on the IT professional to continuously optimize the data that drive that action.

    — Bhanu Singh