devopsy.kinsta.cloud

Tag: scalability

  • The IaC Weight on DevOps’ Shoulders

    The IaC Weight on DevOps’ Shoulders

    It’s often challenging for established tech companies to adopt modern DevOps practices around infrastructure-as-code (IaC). It’s not so much in adopting the syntax, but rather the DevOps culture that IaC is part of for engineering organizations.

    Organizations often struggle to transition from traditional operations models to collaborative DevOps culture, which is a byproduct of transitioning infrastructure operations to code.

    In many cases, legacy tendencies persist, while IaC duties stay siloed within ops-centric DevOps teams—which is the exact opposite of the benefits of leveraging powerful technology like infrastructure-as-code. What this ultimately creates is bottlenecks in development velocity and product iteration.

    With all this in mind, it begs the question: What changes can enable a smoother IaC journey?

    The Pitfalls of IaC Silos

    Having DevOps teams shoulder the entire IaC burden is the exact anti-pattern we’re looking to avoid when truly transitioning to DevOps culture in our organizations. What many organizations start to experience is these familiar pain points that start to emerge:

    1.) Knowledge Gaps Widen

    Developers—by nature of their day job as software engineers—often lack access, visibility and context into infrastructure changes. Yet, with more and more infrastructure requirements shifting left, they are left (pun intended) with insufficient capabilities to manage these growing needs.

    This is compounded when they are faced with minimal cloud permissions. Limited permissions make it difficult to experiment and familiarize themselves with cloud tools and capabilities the way they would with code and its parts of the stack.

    Meanwhile, DevOps teams—as the exact mirror image and the nature of their role in the organization—don’t grasp application needs and objectives. This often results in a widening chasm of broken communication and collaboration around IaC initiatives.

    2.) Collaboration Suffers

    A common tool that exacerbates the problem is a ticketing and support platform. Developers, as their modus operandi, lacking a shared understanding of the domain, will often resort to opening tickets to achieve their goals. This creates a culture of inefficient cross-team communication via ticketing systems.

    This doesn’t mean the ticketing platforms are the problem, it just results in inefficient communication methods, as developers don’t always know what questions to ask.

    What this does mean is that DevOps engineers operate largely on instinct, being mostly in the dark on the real application requirements that can’t all be communicated in a simple Jira ticket. This results in final solutions that rarely satisfy needs on the first attempt and take longer to deliver.

    3.) Releases Slow to a Crawl

    DevOps teams hold the responsibility for conducting and orchestrating release engineering, on top of provisioning environments, maintenance, monitoring and other tasks that fall in the DevOps engineering domain.

    This is why DevOps tasks can become a bottleneck, as they not only dictate deployment timing (due to important considerations of uptime and client disruption, among others) but are also tasked with juggling infrastructure changes across many and multiple environments.

    This causes unpredictable delays in provisioning dev environments and other developer-oriented tasks that frustrate developers and stall production updates, and delivery velocity.

    The Obstacles of Change

    Recognizing these inefficiencies and shifting these responsibilities left became the popular way for DevOps leaders to distribute IaC duties. This allowed developers to gain sufficient autonomy to choose the solutions for their needs, while DevOps teams provided guidance and oversight for how to practically achieve these solutions.

    However, like all things shift left, this comes with its own set of unique challenges. Adopting the DevOps paradigm faces fundamental obstacles that add further complications and complexity to these cultural transitions, including:

    • Uncontrolled sprawl
    • Ballooning costs
    • Security concerns

    On the one hand, distributing the IaC load lessens the burden on the DevOps teams, but the downside is that it becomes difficult to understand which resources are actually in use and which have been temporarily created for testing purposes.

    With many owners creating resources on demand, once they are no longer needed, these leftovers create confusion around dependencies and make cloud platforms disorganized and difficult to maintain.

    Just like enabling more hands to touch IaC creates greater sprawl and disorder, more users with less governance invite careless sprawl in terms of costs as well. This often results in duplicate and unused resources accumulating, wasting budgets that are currently tight, and every penny counts. With a lack of automation and oversight, environments grow messy and expensive.

    The sprawl issues can also impact security, as expanding permissions raises valid security concerns that are intensified when clouds become disorganized and difficult to maintain.

    Well-intentioned developers may misconfigure resources or expose sensitive systems, and without proper methods to manage drift or misconfiguration, this can pose real risks to organizations and systems. Another important aspect that also increases with less oversight is intentional insider risk.

    From Silos to Shared Responsibility

    OK, that must have been really disheartening—and you’re probably thinking you should run for the hills before attempting any transitions to shared responsibility.

    But wait—don’t panic yet. It is possible for teams to redistribute IaC responsibility while avoiding these pitfalls.

    Below are some key factors to help you make the transition smoother in the short and the long term:

    • Automate everything—If DevOps practices have taught us anything, it is that manual cloud environment maintenance doesn’t scale. Automation has been the backbone of much of the evolution we have undergone when it comes to infrastructure operations, and this has been powered by the programmability and everything-as-code mindset—from IaC to policy-as-code (PaC). This automation-based optimization makes it possible to reduce neglected resources through regular and automated analysis and response. What you get is cost savings and the prevention of runaway sprawl.
    • Empower through governance—Remember when we spoke about automation and governance? Well, programmatic policy enforcement is a good way to constrain unwanted resource proliferation without manual intervention. Governance comes in to handle auto-remediation of misconfigurations or other anti-patterns that do not align with automated policies, which simplifies environment management at scale. Importantly, governance, if done right, can be a tool to empower teams, break silos and encourage autonomy. Having the right guardrails in place is how you instill confidence and provide helpful direction.
    • RBAC for the win—Role-based access control (RBAC) limits permissions to appropriate needs, establishing important guardrails that can help prevent some of the issues we spoke about above, including sprawl and security concerns.

    RBAC will enable developers to gain the necessary freedom while minimizing security risk. Minimum required access encourages responsibility and constraint.

    Now that you have a solid grasp of the principles, it’s time to put them into action!

    Fortunately, there are numerous solutions at your disposal. You can jump in and build them yourself using capable tools like Jenkins or Atlantis.

    Alternatively, for a more comprehensive approach, there are excellent commercial IaC management tools on the market. These can assist you in reaching your objectives swiftly and with greater efficiency.

    An Evolutionary Vision for IaC and DevOps

    As technology complexity increases, no single team can handle infrastructure alone. Dev and Ops must unite abilities to shepherd modern systems and the underlying code powering them. This requires closing knowledge gaps through collaboration and communication

    With a thoughtful process, IaC automation, governance guardrails and the right access policies, tech leaders can guide this cultural transition smoothly without exposing the organization to unnecessary risk and spending.

    When done right, Shared IaC responsibility will boost velocity, agility, and autonomy—the trifecta of crucial traits that quantify and qualify elite engineering teams and differentiate companies in a competitive landscape.

    Interested in learning more about env0 to manage your IaC at scale and confidently?

    Stop by Booth L12 at KubeCon EU to speak with an expert or see a demo!

    To hear more about cloud-native topics, join the Cloud Native Computing Foundation, Techstrong Group and the entire cloud-native community in Paris, France at KubeCon+CloudNativeCon EU 2024 – March 19-22, 2024.

  • Dependability is Crucial for Real-Time Experiences

    Dependability is Crucial for Real-Time Experiences

    Millions of people around the world experienced a major disruption after an October 2021 outage saw Facebook, WhatsApp and Instagram go offline. A few days prior, the popular workplace collaboration platform, Slack, also went down for some users. And in the last few weeks of 2021, Amazon’s AWS caused considerable global chaos after significant networking issues in its busiest data center, US East 1, disrupting streaming services such as Disney+ and Netflix.

    Downtime can be costly, causing organizations to lose $5,600 per minute and up to $300,000 per hour, on average, according to Gartner. But it’s not only financial and reputational loss that businesses need to worry about. Outages also seriously downgrade—and in the case of real-time experiences, completely disrupt—the customer experience.

    At a time when consumers are demanding more live and interactive digital experiences, organizations can ill afford disruption or downtime, as customers quickly move to other services that ‘just work’.

    It’s not just the capacity within a real-time system that matters, but also globally distributed failover capabilities when something does go wrong. While organizations could try to build their own failover mechanisms, it’s extremely difficult to failover so services continue seamlessly, regardless of the status of its underlying infrastructure. Simple failover systems not only take longer to resume but typically suffer data loss during the time that a region is disrupted. Businesses need to know that even if, say, AWS suffers an outage, they won’t be affected.

    This is why it’s vital that organizations get it right when looking to develop their own real-time infrastructure. Managing tens of thousands if not hundreds of thousands of concurrent connections while synchronizing data in real-time means that if anything does go wrong it’s immediately obvious to users. If tech behemoths like Amazon and Facebook can’t get it right, what chance do organizations with far fewer resources have when building and managing their own real-time infrastructure?

    The Steps to Success With Real-Time Infrastructure

    Regardless of whether it’s for synchronized multi-user collaboration, audience engagement or instant updates, developers have a hard enough time developing even the most basic of real-time infrastructure, let alone making sure it’s dependable. There are four key elements to achieve dependability: Predictable performance, data integrity, reliability of service and high scalability.

    • Predictable performance: When we think of performance we usually imagine more computing power or speed. But when it comes to real-time, it’s not just about minimizing latency and bandwidth, it’s about having a predictable system. If developers have global latencies and bandwidth within certain operating boundaries, it provides certainty in uncertain operating conditions. This makes it easier to design, build and scale features around this certainty.
    • Data integrity: As we all know, the internet can be an unreliable network. Users abruptly disconnect and reconnect all the time, resulting in data that gets lost, is delivered in the wrong sequence or that is sent multiple times. As a result, organizations must ensure their infrastructure can guarantee high integrity without sacrificing anything else.
    • Reliability of service: The live nature of delivering updates in real-time requires total reliability. Outages can cause distrust and annoyance from users. Organizations could find their reputation taking a hit or customers quickly jumping ship if reliability proves to be a recurring issue. To prevent this, organizations should develop infrastructure that’s fault-tolerant and redundant at both a global and regional level so they can deliver a great experience even if there are multiple failures.
    • High scalability: More often than not, delivering an experience in real-time can result in unpredictable traffic; for example, a huge number of people wanting sports updates for a World Cup game. If thousands of devices connect over a short period of time but capacity can’t scale up quickly enough, then an outage will occur. To avoid this, developers must use infrastructure that can be elastic and meet changing levels of demand. Otherwise, the user experience is seriously affected and real-time capabilities crash when the network is swamped with connections it can’t cope with.

    While organizations often build their own infrastructure with the best of intentions, it’s easy for them to run into trouble. Most lack the internal expertise, resources and time; plus, going it alone often results in a huge technical debt. It’s also a massive undertaking, due to the amount of ongoing complexity and cost involved.

    Third-Party Partnerships Enable Dependability

    Unless organizations have the resources, finances and expertise of the likes of Amazon or Google, then building your own real-time infrastructure is just setting yourself up for failure. Instead, they should save on development cycles and focus on generating revenue more quickly by offloading their real-time infrastructure needs to a third party. This will mean organizations can design without limitations and focus on solving the challenges that really matter to them while putting in place real-time infrastructure that is as dependable and future-proof as possible.

  • The New(ish) Complexity

    The New(ish) Complexity

    The API economy and microservices architectures, led by Agile and DevOps, have introduced a massive amount of hidden complexity. I call it hidden complexity because we have a tendency in IT to focus on the problems in front of us, resolve them, then focus on the new problem in front of us. This leaves technical debt in little piles behind us, like the bits that fall off the ends of a snowplow blade (or the asphalt that squishes out the end of the steamroller for our southern friends).

    I was thinking about this when I was looking into how a majority of you handle scalability. The layers of complexity for scalability are surprising. In the past, we invested upfront to be prepared to scale to the levels we need. Today we allow expenses to grow as users grow. That is a great model if users are tied to revenue. If there is any fungibility at all in the correlation of users to revenue (and there almost always is), this is a potential disaster waiting to happen. “Our traffic spiked 200%, but revenue only grew 20%,” is pretty much the last thing your business leaders want to hear. Because in the “Pay as you grow” world, a 200% spike in users means a significant spike in overhead. How much overhead varies pretty wildly, of course, so I’ll just say “Guaranteed and significant” and leave it at that.

    This is a known bit of complexity, but those of you I’ve spoken with have done little to hedge against it. That’s … interesting. How many other hidden land mines are out there, being ignored by us?

    What can you do? That greatly depends upon your business. with internal systems, you have a lot of say in usage. External systems are where this is potentially an issue. You can pay your provider to block DDoS, so at least you’re not paying for that bandwidth if that happens—and most of the people I spoke with really do have some form of DDoS protection that stops traffic before it becomes a fee-generating hit. But for actual growth, the best you can do is get with business leaders and determine what influences the correlation between users and revenue. What makes those people upping your traffic actually spend money while they’re upping your traffic? What can you do to convince more of them to spend money? What services are you not offering that might be useful to those customers and not significantly increase overhead. Basically, this is the modern equivalent of the conversion problem. They’re on your site, how do you convert them to paying users of your site.

    We really can’t get more detailed than that—you know your business, and a media site is a different model than a retail site, for example—so you’ll have to develop beyond that. But have a plan that does what you can to make sure that when expenses go up, revenue goes with it.

    And keep rocking it. We’re outside the historical IT realm here—because you’re all rocking it so well that we have new things to worry about. Give those customers rockstar apps to work with and figure out how to make them paying customers. Develop those bonds with business leaders; you do indeed need each other.

  • DevOps World: Forget Doc, it’s All About Dev

    DevOps World: Forget Doc, it’s All About Dev

    This year for DevOps World, we’re heading back to the future. Doc Brown won’t be there with his souped-up time machine, but plenty of devs will. As the world is coming to realize, developers are keeping everything running, from the apps we use on a daily basis to check the weather, bank online or order food to the self-parking features on autonomous cars and artificial intelligence that’s augmenting human intelligence. Software and technology are the future, which means the future starts with the creative mind of the developer. 

    The theme of this year’s DevOps World show is “Building the Future of Software, Together.” For developers, this cannot ring more true: Across the globe, dev teams are working to code new software pipelines that will power the world’s next great innovations. We’ve already seen voyages into space and breakthrough life-saving medicines this year—what more do developers have in store for us?

    As we look ahead to DevOps World 2021 on Sept. 28 to 30, we are putting the focus on developers by offering a number of sessions to help sharpen their skills and prepare them to build the future of software. In particular, we are looking at some of the recent challenges that have been affecting code, including cybersecurity, scalability and CI/CD best practices. 

    On the security front, we’ve seen a number of cyberattacks dominate news headlines this year. As a result, companies are hyper-focused on securing their software pipelines and ensuring business and customer data are safe. Developers now need to double as the first line of defense for their organizations, ensuring their code and pipelines are as secure as possible. We’ve geared a number of sessions towards helping developers sharpen their security chops—from CloudBees customer success stories to tips for building more secure code. 

    As DevOps professionals know, software today is about continuous everything. We’ve designed a number of sessions to share best practices for CI/CD, progressive delivery and balancing speed, innovation and safety. Imagine if the DeLorean didn’t need the plutonium and just ran continuously—that’s what we’re hoping developers will learn and take away from these sessions!

    Lastly, scalability has always been a hot topic in tech. Startups continue to pop up everywhere and as these companies grow, they need to scale accordingly. We want developers to be scaling pros when they need to be, so we have a number of sessions (including keynote presentations) on scalability. 

    DevOps World is the event for developers looking to sharpen their skills, learn new abilities and network with other dev professionals. As we look to build the future of software together, the people making the magic behind the code will be driving changes and innovation, and we want to celebrate that.  While Doc Brown may be a fun character who goes back to the future, it’s dev where the future truly is. Register for DevOps World now!