devopsy.kinsta.cloud

Category: Doin’ DevOps

  • Is SASE the DevSecOps of Infrastructure Management?

    Is SASE the DevSecOps of Infrastructure Management?

    DevOps gained traction when IT and Ops (operations) teams recognized that traditional silos and handoffs were leading to inefficiencies and delays in software delivery. While DevOps enabled seamless collaboration and delivered on its promise of speed and efficiency, security remained an afterthought. To overcome the security loophole, DevSecOps emerged, proposing a shift-left approach to security. It emphasizes conducting tasks like security scanning, vulnerability assessment, and compliance checks all throughout the CI/CD pipeline.

    With DevSecOps, speed no longer comes at the cost of security.

    The DevSecOps approach has helped organizations strike the right balance between security and agility in software development. As the infrastructure landscape continues to evolve with the rise of cloud and edge computing, resource-intensive AI and ML initiatives, and distributed workforces, organizations must establish a similar balance between security and ubiquitous connectivity. This is where secure access service edge (SASE) comes in.

    SASE’s Philosophy is Similar to DevSecOps

    As legacy networks — characterized by rigid hardware-based infrastructure — struggled to keep pace with the demands of modern business, SD-WAN emerged as a solution, providing flexibility, centralized management and dynamic traffic routing. However, when distributed computing and remote work virtually dissolved the corporate perimeter, perimeter-focused security tools were no longer sufficient either. Today, enterprises must extend their security capabilities to almost unlimited network access points.

    SASE addresses this challenge by merging networking and security into a single cloud-native framework. It integrates the entire security stack across all edge locations, ensuring secure connectivity between users, applications and resources no matter where they are. Just as DevSecOps shifts security to the left, making it an inherent part of the entire SDLC (software development lifecycle), SASE shifts security to the network edge, making it an inherent part of network connectivity.

    The strategic resemblance between the philosophy behind SASE and DevSecOps is uncanny.

    1. Breaking Silos

    DevSecOps breaks down the silos between DevOps and security teams, promoting collaboration and shared responsibility. In a similar vein, SASE unifies all network and security functions into a single architecture, effectively dismantling the conventional boundaries between the networking and security teams. It allows both teams to jointly manage, monitor, and secure the infrastructure and network traffic through a unified management dashboard. SASE also effectively breaks the data silos that leave security teams with no insights regarding the networking conditions. SASE’s single pane of glass visibility enables networking and security data correlation, broadening each team’s perspective and eliminating tunnel vision for accurate and robust threat detection and an enhanced security posture.

    2. Embracing the Cloud

    DevSecOps aligns perfectly with cloud-native technologies like infrastructure as code (IaC) and containerization technologies like Docker, making it imperative for cloud-first and cloud-ready organizations. On the other hand, SASE is inherently cloud-native. It delivers network and security services from the cloud, providing organizations with the agility and scalability needed to meet the dynamic demands of a distributed environment.

    Traditional back-haul-based connectivity and perimeter-based security solutions introduce latency and coverage gaps. SASE performs network and security functions at the edge, closest to wherever resources and users are. This eliminates the need for disparate on-premises point solutions and extends corporate security perimeter to match the cloud’s global reach.

    3. Agility Through Automation

    DevSecOps emphasizes automation of security tasks, such as security checks, scans and compliance assessments, within its continuous integration and continuous delivery (CI/CD) pipelines. SASE also leverages automation for networking and security tasks, such as dynamic policy enforcement, AI-driven threat detection and response, and network optimization through dynamic routing and self-healing properties. Automation reduces the need for manual intervention, bolsters security, simplifies operational tasks and enables organizations to respond to evolving demands and circumstances faster, crucial for agile environments.

    SASE as an Enabler for DevSecOps

    Just as DevSecOps enhances security within the context of application development, SASE enhances security and efficiency within the realm of networking and infrastructure management. Both approaches share the common goals of integrating security, promoting automation and ensuring scalability. They may apply these principles to different aspects of IT and cybersecurity, but DevSecOps and SASE can work together seamlessly to support modern IT environments.

    Many organizations adopting DevSecOps realize that they lack the necessary infrastructure to support this approach effectively. DevSecOps demands seamless global collaboration and the integration of security controls across the entire IT environment. This is something SASE can help with. It can simplify infrastructure management and enhance security for DevSecOps in several ways. It offers ubiquitous connectivity for seamless collaboration, provides built-in security measures for faster incident response and ensures reliable infrastructure. As such, these are all crucial aspects for achieving the ultimate DevSecOps goal of delivering secure and high-performance applications at speed. With SASE, organizations can set their DevSecOps up for success.

  • Three Strategies for Reducing MTTD and MTTR as Outage Costs Spiral

    Three Strategies for Reducing MTTD and MTTR as Outage Costs Spiral

    According to a New Relic survey of IT professionals, the median annual cost of IT outages has reached an astronomical $7.75 million. Over one-third of the 1,700 respondents said critical business application outages now cost more than $500,000 per hour. In a business environment where most interactions with customers, suppliers and business partners are conducted digitally, downtime has become a problem and an existential threat.

    Most IT organizations carefully monitor two key metrics — mean time to detection (MTTD) and mean time to resolution (MTTR) — to track their success in identifying and remediating critical systems issues. The figures represent, respectively, the average time it takes administrators to identify that a problem has occurred and to remediate the underlying issues. They serve as a benchmark for understanding the organization’s level of awareness of the status of its systems and the speed with which problems can be diagnosed and acted upon.

    There are many ways to improve performance against these metrics, but three stand out as the most effective, as evidenced by New Relic and the survey results.

    Monitor Everything

    The less you know about a problem, the longer it takes to diagnose and fix. The New Relic survey measured 17 different observability categories, such as network monitoring, alerts, log management, browser monitoring and distributed tracing. Across the board, about two-thirds of the companies that used even one of those monitoring tools reported reduced MTTR, with many saying resolution times had fallen by more than 25%.

    One-third of respondents whose organizations implemented full-stack observability—which is the ability to see everything in the tech stack that could affect the customer experience—reported the fewest outages, fastest MTTD and MTTR, lowest outage costs and highest median annual return on investment compared to all respondents. For example, those who have full-stack observability experience a median outage cost that is 37% less than those without.

    As the complexity of enterprise technology stacks increases, so does the difficulty of pinpointing the root cause of an outage. Any monitoring is better than no monitoring at all, and the more visibility you have into applications and infrastructure the faster you can identify and resolve incidents. Strive for full-stack observability around integrated observability data and a comprehensive dashboard.

    Have an Incident Management Action Plan

    The worst time to figure out how to resolve a problem is when you’re in the midst of a crisis. Organizations with mature observability practices also have extensive and documented response plans.

    Your incident plan will be unique to your organization. Small companies and startups tend to have ad hoc response strategies based on the skills within the organization. Large enterprises may use a more formal methodology like IT service management that defines strict procedures and protocols. Companies that have embraced modern DevOps and Lean principles often use a hybrid approach that depends heavily on teamwork and collaborative problem-solving. All of these approaches can work if the team fully understands the chosen strategy.

    Document your IT architecture, identifying known vulnerabilities and outlining procedures for responding to the most common types of incidents. Plans should cover obvious disruptions like fires and power outages, but you should also think about “unicorn” scenarios such as a rogue query that ties up a database server or a communication outage at a key supplier. Mature observability teams devote a portion of their resources to constantly testing new and unpredictable scenarios and use chaos engineering to randomly inject problems in a controlled manner to see if systems respond as expected.

    As you develop incident response procedures, document the processes in runbooks, which tell responders exactly what to do when a specific problem occurs. This not only creates a database of organizational knowledge about response strategies but also helps with onboarding new employees and covers for skills gaps that occur when team members are unavailable or leave the company.

    Fix for the Future

    Technical debt is a long-term obligation that is created when a short-term – and incomplete – solution is applied. You should aim to minimize it in your observability operation.

    When a problem occurs, the temptation is to fix it as quickly as possible. Human nature is to move on to the next task without taking the time to go back and understand why a problem occurred. “Quick and dirty” fixes are an invitation to larger problems in the future. Your observability plan should include a forensics stage that documents root causes and the fixes that were applied. Even if it isn’t convenient to implement a long-term solution immediately, the documentation serves as a guide for implementing a permanent fix down the road.

    Building a resilient IT architecture is a marathon, not a sprint. The more rigor you apply to identifying and resolving problems when they happen, the better prepared you will be to prevent their reoccurrence and the more informed your team will be about the resources they manage. Reducing your MTTD and MTTR will not only improve the bottom line of your business, it will ensure that your customers have an amazing digital experience.

  • Observability: The Central Watchtower Your All-Seeing IT Needs

    Observability: The Central Watchtower Your All-Seeing IT Needs

    If there’s something you need to keep secure, it’s natural to want to keep it sealed away under lock and key. However, even keeping something in a vault, far from prying eyes and ill intent, may not be enough to offer peace of mind. After all, how can you have the peace of mind of knowing it’s safe if you don’t regularly check to ensure it hasn’t bolted, been burgled or broken down?

    Now, multiply the volume and value of things being monitored tenfold — and you get a sense of the daily inefficiencies and anxieties IT professionals battle.

    Instead of jewels, these teams safeguard, manage and maintain something far more valuable: The data that underpins everything from business transactions to system performance to threat detection to service delivery and beyond. To make matters more complicated, today’s digital ecosystems are more complicated, distributed and disparate than ever before.

    Databases are critical to managing a successful IT department — and it’s time to discard the widespread industry’s perception of them as an unknowable black box. To be sure, databases represent the most difficult component of the IT ecosystems to observe, tune, manage and scale. But the database isn’t a Magic 8 ball in terms of its internal processes; We no longer have to shake the mystery toy and accept one of the six answers it’s designed to spit out.

    The fact of the matter is that database specialists and IT teams need a clear view of database performance telemetry if they want any hope of maintaining the health, stability and scalability of their services.

    Observability Tools

    This is where observability tools can play a transformative role. Taking its cues from the all-seeing panopticon watchtower, observability eliminates the dark corners of the database dungeon by providing a comprehensive view across the full cloud-native, on-premises and hybrid technology stacks.

    The word panopticon derives from the Greek word panoptes, or “all-seeing.” The panopticon model features a central, lighted guard tower at the center of a circular room — allowing 360-degree lines of sight that allow the guards to observe every surrounding cell from a single, centralized vantage point.

    English philosopher Jeremy Bentham had no way of foreseeing the challenges of today’s database professionals when he first conceived of the panopticon in the 1700s. The concept would lay the blueprint for a prison system designed around one core concept: Enabling the minimum number of guards to effectively monitor the maximum number of inmates. Bentham’s system also works as a strategy for ensuring the health and performance of your organization’s critical databases.

    The panopticon was designed so that every cell could be monitored from one central point — ensuring not only better and more straightforward security but preserving resources in saved time, manpower and effort expended. Think of your ITOps, database, DataOps and DevOps teams as the wardens of your IT performance. Traditional monitoring can be likened to patrolling a traditional prison: Rows of cells facing out into a hallway that is patrolled on a schedule or used to spot and address misbehavior as it’s noticed.

    How can they be sure they’re focusing their efforts in the right place? Or trust that Cell Block A won’t stage a breakout while they’re breaking up an argument in Cell Block B?

    In short: They can’t. Odds are, the team is either running from outage to outage in an everlasting firefighting mode or relying on random spot checks for something as vital as the health of our IT systems.

    Observability, as opposed to monitoring, puts teams into that central panopticon watchtower, allowing them to not just see everything, but to see it within the context of the bigger picture. This comprehensive visibility and always-on observation enable IT teams to identify critical issues as they occur, even those caused by complex dependencies between the database, operating system, storage subsystem and network.

    Database Performance Issues

    Furthermore, database performance issues are a starting gun for costly bottlenecks or serious outages that can handcuff your company’s ability to compete or grow. Without complete and precise database monitoring and observability, IT operations struggle to accurately determine an application’s root cause of performance issues. This increases the risk of downtime, data loss and poor customer experiences.

    Discovering the root cause of performance issues is quite literally a system-critical undertaking, whether you’re simply running your enterprise, scaling your operations or deploying new code. Observability also offers teams the rare opportunity to get ahead of performance issues since you can see performance anomalies in advance of a service disruption rather than relying on real-time remediation.

    With traditional methods, DevOps and IT teams must manually analyze the data presented to them, correlate it to the problem, and locate the error before they can finally begin addressing it. In contrast, observability collects data to provide information on what’s not performing as expected and why. This allows teams to take a proactive approach to solving downtime and bottlenecks and proactively preventing them.

    In the software world, that translates directly to more time spent where it matters: Developing new business value within your applications and infrastructures, fueling innovation and exceeding customer expectations.

    Given how complicated the modern database has — and continues to — become, arming your enterprise with the right observability solutions can provide a boost to efficiency and performance and pardon your teams from a sentence to time-consuming, mundane and error-prone work.

  • How DevOps Teams Can Unlock Cloud Savings and Improve DevEx

    How DevOps Teams Can Unlock Cloud Savings and Improve DevEx

    Can DevOps engineers unlock cloud cost savings for their organization that are incremental to what cost optimization tools offer (and improve the developer experience in the process)? Absolutely. Allow me to illustrate how.

    Reserve instance buying optimization and auto-stopping are proven tactics to reduce waste, but are those the only forms of waste that exist in your cloud budget? Of course not.

    Think about all the times environments have been brought up and then stayed up longer than required. Consider all of the times a commit changed the way data is stored, which spiked the cost to operate. Or how about the times when a commit changed how memory was used, which resulted in unexpected autoscaling?

    Collaborative Cost Management

    So the question becomes, what is the most efficient way to identify and root out additional savings while also balancing governance and agility? Bottom-up, collaborative cost management may offer a solution.

    A bottom-up approach to cost management involves pushing knowledge and functionality to the edge of the application and data engineering teams that own and operate code every day. This does not mean ceding centralized cost control to developers but rather arming them with timely, actionable insights inside their everyday workflow and, where appropriate, enabling action on a self-serve basis.

    How does this work in practice? It starts by enabling engineering teams to see the costs of the various services and environments they own on a timely basis. This helps teams understand when spikes occur and provides a path they can follow to understand why the spike occurred.

    That path can involve bringing different pieces of information together for engineering teams. They’ll want to see the costs of the individual resources that power their service or environment during the selected timeframe to see what spiked and maybe even drill down further to see the cost drivers of that individual resource. They’ll likely want to see deployments and commits, at a minimum, and maybe even explore observability data as well during this timeframe.

    Once they’ve identified a root cause, to the extent it involves needing to change a configuration setting of a resource, teams may want to understand a resource’s dependents, like other resources that are linked to it as well as any other environments and services that depend on it to size up the blast radius of a potential change. From there, they may wish to view config settings for the resource.

    Collaborative Discussion

    Finally, you want to give teams the ability to take certain actions on their own and, in other cases, foster a collaborative discussion between engineering and DevOps. You can accomplish the former via self-service actions, which are simple no-code workflows in which a developer can see if the DevOps team has allowed an action to be seen by a specific team for a resource via a role-based access control (RBAC) model. These workflows connect to your existing infrastructure-as-code (IaC) or CI/CD tools to orchestrate the desired change, leveraging golden paths defined by DevOps. This approach alleviates developers’ cognitive load and TicketOps for the DevOps team. For various changes that require coordination with DevOps, your engineering teams can have a collaborative discussion on a timely basis thanks to the fresh, actionable information you’ve made available to them.

    So what does it take to actually implement this vision? It’s easier than you might realize.

    Chances are, your existing cost optimization tool cannot provide cost visibility at the level of granularity and freshness developers need – daily cost insights at the service (or data pipeline, ETL job, etc.) level. That’s okay, though. Collaborative cost management is an incremental tactic that can still leverage some of the insights in these tools.

    The easiest way to facilitate this competency across your enterprise is via an internal developer portal. Internal developer portals are living sociotechnical knowledge maps that connect various data from your tools, clouds and teams to your services and the teams that own them. They are the perfect conduit to slice cloud cost data at the service level and put it in front of application and data engineers in their everyday workflow.

    Backstage, an open source framework to build a developer portal, offers a cloud cost plugin, though it involves a time commitment to set up, and the information it makes available to your developers may be limiting in the context of this scenario. For instance, your team may be unable to see per-resource costs, individual resource cost drivers, blast radius information and config settings. You must also expend effort wiring actions to support resource changes.

    You could also try building a BI dashboard for your team to accomplish this. This would require ingesting daily cost feeds from your cloud and then allocating the cost of each resource to each environment and to each service leveraging proprietary resource<->environment<->service mappings you would build and maintain as your system evolves. You would also need to build scripts and leverage IaC and CI/CD tooling and teach your engineers how to discover, configure and execute these actions.

    However you choose to get to this destination, collaborative cost management is a powerful emerging competency that can lead to meaningful savings and improve the developer experience by empowering your teams with the knowledge and functionality they need to manage costs on a proactive and, where appropriate, self-service basis.