devopsy.kinsta.cloud

Tag: scalability

  • DevOps World 2021: 3 Sessions for Successful Scaling

    DevOps World 2021: 3 Sessions for Successful Scaling

    Your business has grown from a startup of five people to a full-blown organization of 100 … congrats! But, now what? The practices that worked for a smaller company may have worked as you first started growing, but now it’s time to scale operations to fit your new needs. Just like a kid outgrowing their clothes, your organization needs a new “wardrobe,” and scalability is key for long-term success.

    This year’s virtual conference—scheduled for Sept. 28-30—will offer a number of breakout sessions, panels and keynotes focused on scalability within enterprises. If you’re ready to scale within your business, here are a few sessions from this year’s conference to help get you started.

    From Startup To Enterprise—Sustainably Scaling

    Many organizations struggle to scale DevOps culture as they grow. Now it’s time to say goodbye to the startup phase and hello to thinking like an enterprise.

    Alston Rousseau, director of software delivery at Campspot and Kailene Cloud, director of talent acquisition at Bonusly, will walk attendees through the investment needed to sustainably scale operations, development and culture. Plus, they will share tips for finding more value in existing commitments to money spending, time and diversity in DevOps and how this approach applies to organizations with and without high compliance requirements.

    Scaling up Car Configuration: From a Statically Hosted Front End to Universally Rendered Services

    Over the past year, tech company Code Orange made the transition from a traditional API/SPA split to a much more technically challenging server-rendered SPA with full cache handover.

    In this breakout session led by Niels Hoiting, a developer at Code Orange, attendees will learn more about Code Orange’s transition from API/SPA split to server-rendered SPA while analyzing the pitfalls that can arise in the transition and how to optimize compute spend to best handle a specific workload.

    Leveraging CloudBees to Scale Satellite Ground System Emulation in Multifaceted Virtual Environments

    Aerospace and defense technology manufacturer Raytheon Technologies has successfully implemented CI/CD pipelines for the test and near-operational integration of both software and hardware for a global satellite ground system.

    In this breakout session led by Casey Jackman, deputy of GPS Product Factory at Raytheon, attendees will discover how Raytheon uses simulated satellite control stations in virtual private clouds for CI, test and check-out. Additionally, Casey will share how the team met challenges in initial pipeline generation between multiple disparate target environments with AWS and on-premise data centers.

    Think it’s time to scale your organization? Register now for DevOps World 2021 to get started.

  • The Increased Role of Colocation Providers in Data Centers

    The Increased Role of Colocation Providers in Data Centers

    You would be hard-pressed to find something more complex than a modern data center. A state-of-the-art building stocked with all the necessary software for a major company might run into billions in costs if you decide to build and operate your own from the ground up.

    Even though many pundits had predicted data centers would become a thing of the past due to the ever increasing adoption of SaaS, as we previously predicted, data centers are still popular with companies and evolving for 2020. One phase of that evolution is for businesses to box up all of the gear in their data center and install it with what is called a colocation provider, or “Colo” if you’re looking for a snappy acronym.

    “Going Colo” refers to renting space in a building, usually a rather large building, and moving all your physical servers and network hardware there. You’ll have neighbors who are doing the same thing because this is just a fancified way to say you’ll have a landlord.

    The Colo market is getting big in a hurry. The global market was $31.5 billion for this type of data center arrangement in 2017, but is expected to expand to at least $62.3 billion by 2022. What a company, large or small, has to ask itself is if this type of arrangement is a good thing for DevOps. Let’s ponder that.

    It’s Cheaper When You Don’t Own the Building

    If you remove the possibility that you could be thrown out of the space at a moment’s notice (a worst case scenario that a good rental agreement should address), the idea is a no-brainer from a cost savings perspective.

    For an enterprise, let someone else take on the risk of construction and infrastructure maintenance. For a small-to-medium-sized business, the cost of building and monitoring a modern data center makes it impossible.

    The unfortunate reality is that, when you do own the building, someone has to take care of it. In a perfect world, every company would have a dedicated IT staff; if you’re Microsoft, Apple, Google or Facebook, that certainly is the case.

    For much of the tech world, the choice to maintain a data center onsite often results in personnel being frequently dragged out of their role in DevOps and asked them to do something related to the data center, often a task they are not qualified for. This creates holes and inefficiency in the development process. That’s a bad thing. 

    Good Infrastructure Means Better Uptime

    Guess what’s also bad for DevOps? The team’s least favorite scenario—besides an unexpected meteor strike—would be system crashes and downtime related to hardware issues. Downtime can be incredibly expensive for a multitude of reasons besides the obvious—employees sitting around twiddling their thumbs. There is also the scenario of actual lost income. As Amazon found out the hard way, the number can be astronomical.

    With no internet connection, business payment systems, no matter how sophisticated, are likely to be non-functional, though some will queue payments and run them through when the system goes back online. This is why it’s a good idea for businesses to find ways to speed up transactions from clients and customers, such as automating invoices and customizable invoicing tools that allow multiple options for straightforward payments.

    This is also another area where going Colo can be a huge help to development. Think about it. In return for scratching a monthly rent check, you get diversified and redundant power sources, the most reliable internet on the planet and enough bandwidth to strangle an elephant.

    The bottom line is that moving equipment to a Colo space should be accompanied by a significant reduction in downtime. If it’s not, you need a different landlord. Downtime is the enemy of product/service development progress. A modern Colo arrangement means you don’t have to spend time running down to the local Dollar General in search of an extension cord when one flames out. Okay, you really shouldn’t be using extension cords from a dollar store in your data center, but you get the idea. 

    Secured Data and Premises

    The only type of downtime worse than that caused by insufficient infrastructure is when data is breached either through hacker online trickery or physical penetration of the premises. The latter happens more than you might think. Bad guys decide to go old school and break into a Colo center in the middle of the night to physically drag servers out of the rack and spirit them away.

    With the intense global focus on thwarting hackers, DevOps is already in the process of evolving into DevSecOps as we speak. The reason is simple. Development teams are realizing that a focus on security from the start reduces the chance that a project gets interrupted mid-development resulting in—you guessed it—downtime.

    With a Colo provider, let them worry about security.

    In a Word, Scalability

    With the everyday lexicon expanding to include cloud storage possibilities, the idea of scalability is at least somewhat familiar to the masses and certainly to businesses.

    Google actually started out in a colocation data center before their data collection grew to approximately the size of the Milky Way galaxy. For IT in general and DevOps in particular, scalability is the Holy Grail because no company expects to stay the same size forever.

    A smart DevOps team, especially one that works in any of the growing number of SaaS business models, will put the bug in management’s ear about the benefits of Colo long before the need arises.

    The worst time for an upwardly mobile small or medium-sized company to start thinking about making the switch is after you’ve already outgrown present resources. For example, when working with inadequate bandwidth, a peak time data spike can slow development as local computers struggle to find enough bandwidth to function.

    Another dirty little secret is that Colo providers typically get better internet rates than a single company, so there’s that. With the cost of a spike spread across all tenants, the actual cost to you will be considerably less than trying to manage your own data.

    Final Thoughts

    The bottom line is that the service provided by a Colo center should reduce the strain on a company’s IT budget and resources. Common services provided to a tenant are support personnel, who monitor and troubleshoot systems and issues before they grow into a big, fat calamity. Money you don’t spend providing basic service to an onsite data center is money you can funnel toward doing your company’s main work.

    There’s one more thing a Colo site provides that shouldn’t be undervalued, and that is peace of mind, because you have removed much of the stress, cost and personnel drain that accompanies running your own data center. Farm it out. Let someone run it who hires staff with the particular skills needed and has a vested interest in keeping you happy.

    — Gary Stevens

  • DevOps at Scale: Winning Strategy for Modern Enterprises

    DevOps at Scale: Winning Strategy for Modern Enterprises

    DevOps is a word that has generated massive interest over the last few years among the software-powered organizations and developers, yet many people don’t fully understand what it actually means. One thing is for sure, as cool as it sounds, DevOps is not some kind of top-secret developer army group, job profile or even a tool. It is a wonderful ideology that any firm can help itself to unite their tech crew and take the company to newer heights.

    As different departments grow, communication becomes difficult between the teams, mainly development and operations. Since communication is the key to avoid unproductive siloing, the DevOps movement started as a cultural phenomenon in order to bridge this gap between dev and ops. So firms started embracing DevOps principles to get everyone on the same page and create a sense of shared responsibility, and moving forward, firms believe that was the recipe for success.

    Where Does DevOps Come into Play?

    Automation is one of the things DevOps is most known for. Automation reduces the amount of work, and smaller batches of work are easier to understand, commit to, test, review and know when they are done. These smaller batch sizes also contain less variance and risk, making them easier to deploy and, if something goes wrong, to troubleshoot and recover from.

    With DevOps best practices like CI/CD, it is easy to understand which code has created bugs and make possible corrections wherever necessary. 

    Just like lean development, through automation and agile practice, firms can get feature development very close to single-piece flow, and this provides value to customers quickly and continuously by delivering features as quickly as possible. 

    More traditionally, DevOps is referred to as a means to break down the walls of confusion and chaos between the dev and ops teams and smoothen the flow of communication. In the DevOps model, development teams develop new features, and operations teams keep the system more reliable for customers, stable and running smoothly.

    Today, we will see some important elements that help firms to embrace DevOps with ease.

    Start a Cultural Shift, a DevOps Initiative

    This should flow from top to bottom, company management leaders should start the discipline of embracing DevOps. Usually, it is the CIO who should move the needle and make things start from scratch. A DevOps mindset is very crucial at this point to make sure the teams are all prepared for a change in the culture.

    Gartner predicted that through 2022, 75% of DevOps initiatives will fail to meet expectations due to issues around organizational learning and change. One of the main reasons why DevOps efforts fail is that most of the organizational leaders don’t understand Agile and DevOps. Leadership makes a huge impact on the organizational culture and its way of doing business, hence engineering leaders need to understand why this change is necessary and start a culture of collaboration, culture of experimentation and learning, and eliminate the blame game. 

    Train Your People

    Training is one essential aspect for team leads and the people involved under them so they understand what DevOps is, the fundamentals, and why it is important for the organization to embrace it. Training enriches knowledge, collaboration and communication processes between the people working in development and operations teams. 

    It helps them to get together and get their hands on, test the new technology practices and tools introduced. There are many group courses, boot camps and DevOps experts that can help people get trained in DevOps. It is the firm’s responsibility to set proper expectations and choose the right candidates to train their people. 

    Build a DevOps Plan with a Strategy

    All those engineering best practices that enhance automation, team collaboration, seamless integration and testing that facilitate new ways of software development with ease are key to a DevOps strategy. Firms have to set a common goal, track changes and align teams in a shared environment making them aware of each other’s responsibilities and duties. Implementing Infrastructure as Code (IaC), test automation, integration, deployment and release processes should be a part of it.

    Set Policies and Processes

    The policy is an integral part of any software powered organization. With security being the priority, this has become a supreme topic in the DevOps world. For every company that has adopted DevOps, the policy is critical. As we all know, policy-driven development is a fundamental concept that fits really well into the DevOps model. This model ensures that information security, ops, development practices, language standards, SRE, sysops administration and development, are all aligned with common goals.

    Containerize Applications

    Containers are well equipped with the IaC approach and are used in DevOps to instantly deploy applications across various environments. Docker provides the most popular container technology toolset. Docker image management technologies, such as Amazon ECR, have made it easy for developers to perform IT operations, collaborate with each other, share software and enhance overall software delivery productivity.

    Containerizing legacy applications will reduce the burden of handling the ever-growing overhead costs and it also increases developer productivity. Container deployment can be an effective solution as containers typically start in seconds. The container image holds all the dependencies, meaning there is no need to add environment-specific configurations. The same image is now portable across multiple systems and environments. The main idea is to make applications able to run anywhere, on any machine and at any given time. 

    Make Use of Microservices Architecture

    Decouple the legacy and monolithic application that involves thousands of lines of code into numerous small elements called microservices. With the idea of microservices, a company continues to innovate and stay agile as the products or the platform gets larger. With this approach, engineering teams focus on developing business capabilities and important features autonomously.

    Instead of having to orchestrate a large release after writing hundreds of lines of code, waiting for the perfect time to offer new features, a new business functionality can be added by just adding a new service. Eventually, firms end up creating microservices that can be easily and independently deployed, scaled and managed without having to worry about other parts of the application.

    Determine the Toolchain

    When companies initially start embracing DevOps they feel proud, but they find it hard once they really start working toward it practically. The simple logic for this is: DevOps is fragmented. 

    For many years, firms have tried to embrace DevOps and have found themselves making various mistakes by their custom scripts and logic. From source control to CI, to artifacts management to CD, as soon as one task is completed, the next step should proceed without any lag, and that is how the firms achieve their DevOps automation. But in practice, this doesn’t happen.

    There are many handovers that take place manually—that is not called DevOps. Custom scripts and practices might work temporarily for a startup but when the company grows, taking different tools and writing code to make them talk to each other becomes a complex task. For this reason, recently, JFrog has launched a unified DevOps platform for companies struggling with doing DevOps at scale. 

    Companies around the globe are in a race to innovate and release software and valuable features to their customers as early as possible to stay ahead of the competition. You need a platform to deliver software updates fearlessly from code to the edge. It would help if you had an end to end universal DevOps solution, like JFrog platform, to manage your entire DevOps pipeline. A highly scalable enterprise platform unifies the management of all services in a single pane of glass. The JFrog platform is Kubernetes ready to manage your Docker images and Helm repositories for your Kubernetes deployments. It has everything you need to do DevOps, from artifactory repository management to distribution.

    Continuously Monitor and Share Feedback

    Once we deploy the application in production and make it available for the end-users, the next step is continuous monitoring. This alerts the team if there are any misconfigurations or worse situations of an outage or of any application issues. You can monitor infrastructure health, or if would like to see and analyze logs over weeks and months, you should be able to do so.

    This step provides critical feedback, essential for the team to understand and work on the aspects of the application that require additional work and resources. Continuous monitoring enables you to find issues (if any) before any customer finds them. Having no visibility is just like getting your arms tied behind your back, you can’t really see what is going on. 

    Conclusion

    Other than easy collaboration, improved communication, an increase in productivity and effective feedback loops, the one benefit DevOps will bring is delivering software at speed without compromising on quality. DevOps is an ongoing effort and helps people in the organization fail early and learn early from their mistakes, and it builds confidence. DevOps implementation can look too overwhelming at times, but a prepared approach with a well-thought strategy can take your firm from zero to hero. Given that the teams have patience, persistence and ever-learning attitude, it is possible to scale DevOps to new heights. 

    — Pavan Belagatti

  • Ten Years Later — What Is Cloud Native?

    Ten Years Later — What Is Cloud Native?

    Ever since the term cloud native made its debut in 2010, its usage has grown in popularity. Today cloud native is used as a qualifier for innovative, leading edge applications, platforms and systems or cloud-savvy organizations and processes.

    However, what is cloud native exactly? There are some definitions, there are many opinions, but first and foremost there is a lot of confusion: Is my system already cloud native if I follow a microservice-based architecture? Is my system already cloud native if I run containers? Do I have to change development and operations?

    In this post, we cut through any uncertainty and ambiguity and provide an accurate and concise definition of cloud native systems.

    However, first, we have to discuss a set of prerequisite concepts.

    Application + Platform = System

    Frequently, in our day to day conversations, we use the terms application, platform and system interchangeably. Yet in this blog post, we draw a strict distinction:

     

    • Application: An application is the set of components that are designed and implemented for a specific use case.

     

     

    • Platform: A platform is the set of components that are designed and implemented independently of a specific use case.

     

     

    • System: A system is the combination of an application and a platform. Generally, we refer to an application as being hosted on a platform.

     

    End users do not interact with applications or with platforms only; end users interact with systems. Consequently, we will focus our discussion on system-level properties, not application—or platform-level properties.

    Application + Cloud Platform = Cloud System

    A cloud platform is a service provider that enables a service consumer to request and release resources on demand via an Application Programming Interface (API).

    According to this definition, a cloud platform is any service provider that enables a service consumer to request and release resources on demand:

     

    • Public Cloud Platform: A public cloud platform is a service provider that offers resources outside of its own organization.

     

     

    • Private Cloud Platform: A private cloud platform is a service provider that offers resources within its own organization.

     

    Consequently, a cloud system is any application that is hosted on a cloud platform. However, what is a cloud native system? 

    Responsiveness, Scalability and Reliability

    The distinction of cloud native systems versus non-cloud native systems is drawn along the lines of responsiveness, scalability and reliability. 

    In a nutshell, responsiveness is a system’s ability to meet the users’ expectations. In order to quantify responsiveness, we rely on a set of four related concepts: 

    • Service Level Indicator: A service level indicator is a quantitative observation about the behavior of a system.
    • Service Level Objective: A service level objective is a predicate (a function that yields true or false) on a service level indicator that determines whether the behavior of a system meets an objective.
    • Error Rate: The error rate is the ratio of the number of observations that do not meet their objectives to the number of observations in total for a fixed time interval.
    • Error Budget: The error budget is an upper limit on the error rate.

    Based on the above, we can now define responsiveness of a system as the ability of the system to keep its error rate below its error budget.

    These concepts are best explained with a tangible example: Consider a web application processing HTTP transactions, that is, accepting HTTP requests and returning HTTP responses.

    Service Level Indicators

    Here, we define two SLIs.

     

    • SLI Latency: determines the latency of a transaction, that is, the time when the request enters the system to the time the response leaves the system. 

     

    • SLI Result: determines the result of a transaction, that is, the status code of the response.

     

     

    SLIs are recorded at the edge of a system, here the Ingress component, to exclude any variation that is outside our control and focus on the performance of the system that is within our control. For example, we do not want to include any variation in observed latency or observed results originating in the users’ network connections. However, we do want to focus on the performance of the system once Ingress accepts the request until Ingress returns the response.

    Next, we define two SLOs.

     

    • SLO Latency: For this example, we define the SLO Latency to be met if a transaction’s SLI Latency is less than 250 milliseconds.

     

    • SLO Result: For this example, we define the SLO Result to be met if a transaction’s SLI Result is not in the HTTP 500 range.

     

     

    Finally, we set the value of the error rate’s time interval to a 30-day sliding window and the value of the error budget’s to 1%.

    By definition, this system is responsive if the ratio of the number transactions over the last 30 days that do not meet their SLOs to the number of transactions over the last 30 days in total does not exceed 1%.

    Scalability and Reliability

    At this point we have the necessary tools to define scalability and reliability:

    • Scalability is responsiveness in the presence of load.
    • Reliability is responsiveness in the presence of failure.

    So Finally, What Is Cloud Native?

    With the definition of cloud systems and the definition of responsiveness in place, we can now define cloud native systems: A cloud native system is a cloud system that is scalable and reliable by construction.

    The operator “by construction” is in stark contrast to “by requirement”: The design and implementation of a cloud native application guarantees scalability and reliability within the limits set by the service level objectives and error budgets.

    According to this definition a cloud native application must be able to:

    • Autonomously detect and mitigate the presence of load.
    • Autonomously detect and mitigate the presence of failure.

    Discussion

    This post defined the term “cloud native” as a property of a system, not as a property of organizations or processes.

    The defining criteria of a cloud native system are scalability and reliability by construction. Here, the employment of microservices (an architectural choice) or containers (a technological choice) is not a defining criteria of cloud native systems. If you want to know more, head over to—or better yet, get involved with—the Cloud Native Computing Foundation, a Linux Foundation project founded in 2015 with the mission to build a sustainable ecosystem for cloud native systems.

    — Dominik Tornow

  • The Difference Between Capacity and Scalability Planning

    The Difference Between Capacity and Scalability Planning

    Not only does the tech industry love terms, we love to make them super nuanced. For example SecOps versus DevSecOps, continuous delivery versus continuous deployment, hybrid cloud versus multi-cloud, etc. These terms may seem similar on the surface, but when you dig in they are quite different.

    Here are two more: capacity planning versus scalability planning. Both are critical aspects of keeping applications up and running, but two different activities.

    The reason we invest time in capacity and scalability planning is simple: We want to make sure that system resources such as compute, storage and memory are not the cause of an application outage. It is a response to expected and unexpected increases in application usage, as well as the steady growth of application adoption.

    Here are a few differences between capacity planning and scalability planning.

    Long Term Versus Short Term

    Capacity planning is more focused on the base resources, as it relates to the long-term growth in application traffic. The goal is to have enough resources to handle the steady state of the application without breaking the bank by over anticipating usage. With capacity planning there is a big assumption that the application requires to always have a pool of resources available.

    Automated Versus Static

    Scalability could be a manual task, you see utilization creeping up and you respond with more instances. This is probably more common than not. But I don’t believe it’s the overall objective of most teams. Auto-scaling is the goal, where the system knows on its own when it’s time to spin up new resources. The public cloud providers, instrumentation of kubernetes, etc., help support this. Whereas capacity planning is a more deliberate identification of needs, and then statically setting those resource needs within the cloud.

    Strategic Versus Responsive

    Similar to static versus automated, capacity planning is a more strategic activity whereas scalability planning is a more responsive activity. Usually, scalability comes up during peak usage activities such as black Friday sales or school registrations.

    Capacity planning generally calls for more research and leveraging tools such as load balancers and historic data to understand the best path forward. It also should involve the input from the broader team, especially development to discuss the resource usage of new functionality and application architectures.

    There is a good possibility that scalability planning could be all that is left with some modern applications. The reason this might be true is resource allocation theoretically can be 100% application based, where the resources are always one step ahead of current utilization. In order to embrace this model resources need to be provisioned 100% automatic, which means that all infrastructure needs to be scripted. As more companies adopt serverless components of their application, this is already inherently true.

    Capacity planning is where most operations teams spend their time, but resource utilization is shifting left and scalability planning is becoming more common. Scalability planning has become the responsibility of DevOps engineers, SREs and even developers. The main reason capacity planning will survive the long haul relates to budgets. While the application itself can fully control the infrastructure laid out before it, the IT organization and the organization itself will want to impose an upper limit so there is an indicator of when the cost of running the application on the resources may or may not be an issue. Capacity planning in this case will be purely a cost control mechanism, and negotiated just as any budgets are.

    — Chris Riley

  • Emerging Patterns in SRE Implementation

    Emerging Patterns in SRE Implementation

    First we were told that every company will become a technology company. Then the tides shifted to every company will become a DevOps company. Now the zeitgeist has a new obsession: Every company will become a Site Reliability Engineering (SRE) company.

    Have you been keeping pace? If your company survived the ruthless restructuring of digital transformation and is savvy enough to have surfed the DevOps wave, you might already have heard SRE is the next industry standard on the horizon. And you might be bracing yourself for yet another cycle of emerging best practices. Is the dread setting in?

    Fear not, throwing out the welcome mat for SRE does not require kicking DevOps to the curb. On the contrary, the two are complementary philosophies, which, when harnessed correctly, can usher in a new era of value delivery. Essentially, SRE is a specific implementation of the core DevOps pillars of success that applies a software engineering mindset toward solving traditional operations problems–with a focus on creating reliable and scalable technology. Together, DevOps and SRE work in harmony, striving toward a shared goal of breaking down organizational barriers and delivering better software faster. 

    If you’re ready to give SRE a shot but you are unsure how it best fits within your team’s existing structure, let’s break down the top five team organizational structures that emerge with clients. However, feel free to view these as loose guidelines rather than an exact blueprint. Similar to DevOps, there are diverging views and practices of integrating SRE into a company’s structure.

    Discovering the unique way SRE fits into your team is the most rewarding and fun part of adopting this new system. What matters most here are outcomes. Creating a role for SRE means alleviating some of the pressure on your developers, allowing them to be more productive, and more efficiently and effectively implement the DevOps paradigm. Soon enough you’ll be reaping the benefits of what happens when, as Ben Treynor, Google vice president of engineering said, “a software engineer is tasked with what used to be called operations.”

    The OG(oogle)

    Let’s give credit where it’s due. We wouldn’t have the SRE acronym, let alone the entire discipline, if it weren’t for Google’s innovative ops team. With the aim of bridging the gap between its production and ops teams, Google ops leader, Ben Treynor proposed eschewing the norm of building an ops team solely from system administrators and instead bringing in software engineers who possessed a systems background.

    This model entails creating a dedicated engineering team focused on running and scaling a product or platform. The team should be comprised of highly skilled software and systems engineers who are on-call for the product, and simultaneously directly updating the product code base for reliability and building associated tooling to support the product. Most importantly, the product development team is still responsible for a portion of these operational tasks and may not be allowed to ship new features if reliability falls below agreed upon thresholds (the Error Budget).

    Why Change When You Can Rebrand?

    If your ops, DevOps and platform teams are on board with the SRE philosophy but you either don’t want or don’t need to make significant changes to your product code, this is the approach for you. In this model, your teams can simply rebrand as Site Reliability Engineers, while adopting and implementing the tenets that come with the title. That means shifting the concentration of engineering to focus on improving reliability and scalability. This team would also be on-call, playing a prominent role in the underlying infrastructure, tooling, platforms or day to day support of the product.

    Beware: You need to take seriously the underlying tenets and need for strong software engineering skills on these teams in order to realize the benefits.

    The Inside Job

    Perhaps you’re down with SRE but don’t want it to co-mingle with product accountability. A viable option then would be creating an internal consulting arm–a centralized team solely dedicated to creating and advocating for reliability tools and processes. One of the prominent benefits of this structure is that your team would be able to not only see the forest for the trees but also serve as your SRE gurus, keeping your team abreast of the latest technologies, trends and research in the SRE space. 

    You Get an SRE Engineer!

    For those who want to weave in SRE engineers throughout their entire organization, this approach applies a holistic mindset to the SRE implementation proposition. That means SRE engineers are introduced into cross-functional teams that own the end-to-end lifecycle of a product, from build through decommissioning. From here the road diverges: your choice is to either create an SRE team of engineers who belong to a single capability and are also embedded full time within a product team, or allow product teams to hire their own dedicated SRE engineers. As Cloud and SRE Practice Lead at Slalom Build, I took the former. For us, this approach guarantees bespoke SRE implementation, ensuring right-sized scalability and reliability throughout the product lifecycle.

    SRE? New to Me

    Despite the compounding hype around SRE, it has yet to hit universal name-recognition. If it’s still unfamiliar to you, that’s okay. It’s not too late to raise your hand and get up to speed during this period in which we are still in the early stages of adoption. I assure you that if you reach out to expand your knowledge of SRE and even bring it back to your team, you will be met with enthusiasm and welcoming arms.

    — Rob Cummings

  • The Challenge of Scaling Metrics in Distributed Systems

    The Challenge of Scaling Metrics in Distributed Systems

    Whether migrating from a monolithic architecture or building a distributed system from ground zero, there are many benefits to leverage from a microservices architecture–faster software deployment cycles, enhanced scalability, improved isolation of risk. However, distributed systems are complex and as such, accompanied by many challenges, particularly when it comes to the scalability of metrics.

    The Importance of Observability in Distributed Systems

    In order to realize the benefits of a distributed system, a strategy of data-driven optimization is essential. The paradox here is visibility simplifies complexity, while the complexity of visibility is dramatically increased in distributed systems.

    As New Relic points out in their eBook Cloud Native Is the New Normal, traditional monitoring approaches need to be adapted to meet the demands of decentralized architectures, including:

    • Overcoming the loss of end-to-end monitoring capabilities by learning how to connect the dots across a complex network of environments.
    • Developing tactics to quickly drill down into issues across interconnected parts to reduce mean time to resolution (MTTR).
    • Introducing proactive reliability strategies that focus on measurement first, followed by improvement.

    Finding the Right Balance Between Monitoring Nothing and Monitoring Everything

    Most application engineers leverage some sort of metrics tooling such as Grafana, Prometheus or Graphite to track vitals for their system health. While monitoring is essential, there is such a thing as bloat when it comes to metrics ingestion; and this overload translates to scalability challenges that affect performance, cost and reliability.

    So, how do you know what to monitor? In order to derive the most valuable insights from your system, you must first understand and identify the components, and more specifically which attributes within those components drive key metrics. Knowing this requires an intimate familiarity with how all the pieces work together in your distributed system.

    Google’s Site Reliability Engineering book provides a framework for determining what to monitor in distributed systems. They break signals down into four different categories:

    • Latency: Understand resource requirements and identify bottlenecks.
    • Traffic: Monitor system response to varying levels of load; most useful when paired with other indicators.
    • Errors: Track frequency and nature of interruptions in request/response flow.
    • Saturation: Surface capacity-related issues.

    Even when you’ve identified the most important metrics to track, scalability is still a challenge that must be addressed, particularly when dealing with applications that handle massive volumes of data across a complex application architecture.

    M3: Uber’s Open Source Large-Scale Metrics Platform

    A solution directly addressing the challenges of distributed metrics at scale is M3, an open source project developed by Uber’s engineering team.

    Before using M3, the Uber team sent metrics to a Graphite stack with storage handled using the Whisper file format in a shared Carbon cluster. From there, they created Grafana dashboards to visualize the data and used Nagios for alerting based on Graphite threshold checks via source-controlled scripts. As the volume of metrics continued to expand, manual processes to scale the solution became unsustainable and repeat failures resulting in permanent data loss led them to build a more fault-tolerant, scalable system.

    M3 is focused on providing a reliable storage solution for large-scale metrics over long retention periods, solving inherent single node-associated limitations with Prometheus by offering a multi-tenant store for Prometheus metrics. A key component of this solution is M3DB, the scalable storage backend for M3.

    M3DB, built in Go, is a distributed time series store and reverse index with configurable out-of-order writes. By enabling the storage of real-time metrics over long periods of retention, M3DB can replace a Prometheus cluster. In developing the M3DB solution, the Uber team essentially deconstructed Prometheus into microservices to enable the scaling of each component individually. By taking this approach, M3DB is able to offer better reliability, efficiency and performance.

    At a minimum, M3DB can be leveraged as remote storage for Prometheus (or used to replace any other remote storage solution). There are, however, other features the solution offers, including an external aggregation engine and a separate query engine.

    The M3DB project is still under development with a growing community of contributors.

    To learn more about containerized infrastructure and cloud native technologies, consider coming to KubeCon + CloudNativeCon NA, November 18-21 in San Diego.

    — Molly Wojcik

  • Using Automation to Control Cloud Costs

    Using Automation to Control Cloud Costs

    By building in cost control automation from the very beginning, you can keep your cloud costs low while encouraging innovation.

    Infrastructure automation has become vital for every modern sys admin and operations team. Applications and systems are evolving from single-host mainframes to distributed microservices architectures. This automation has gone from basic operating system installation and setup to full-blown multistep deployments of production code from a single developer’s commit. By automating these mundane processes and eliminating the element of human error, production systems are much more stabile than ever before.

    But why stop at automating deployments? There are other elements that need to be automated, too. One of the main ones is cost control.

    Continually rolling out new infrastructure without ever taking a step back to analyze the cost often leads to the panic-driven cloud-bill-based phone calls from your finance department. Instead, applying automation to your cloud costs can help you to save money, so you never have that unpleasant surprise of a major bill.

    Scaling Up Without Ever Spinning Down

    Developers and operations teams often use infrastructure automation early in application development and deployment processes to get servers and databases deployed and functioning. Modern automation tools aren’t just powerful, but they’re also quick to deploy and fit into your current workflow. This is fantastic, but the problem is the automation effort can taper off once the environments are running—too often, users and teams move on to the next project before figuring out a way to keep costs from getting out of control. Then it’s too late, and users simply accept that money needs to be dumped into the deployment pipeline to keep everything on task.

    Easy-to-use automation is the key to spinning these environments up efficiently. It also can be key for keeping the costs of these environments low. Sure, you may need to keep the production systems scaled up for maximum application performance and customer satisfaction, but what about the test lab, sandbox environment, dev systems, UAT servers, QA deployments, staging hosts and other pre-production workloads? Having giant environments with system sizes that match production can be useful for some testing, but leaving it all running can be easily doubling your cloud costs. This is true for each environment you have, including for elements that are only used for a fraction of the time.

    As your infrastructure automation toolkit grows and evolves, there are a few things you should start building into all of your applications and deployments. These include:

    As this list grows, there’s one more thing you need: continuous cost control.

    By building in cost control automation from the very beginning, you can keep your cloud costs low while maintaining the flexibility required to keep up the pace of innovation. Without this, your costs are destined to rise faster than you intended, which will then cause headaches (and endless meetings) in the future. It may not be coming out of your bank account directly, but saving money at an enterprise organization is everyone’s job. Automating this is the key.

    — Jay Chapel

  • The Beauty of Inheritance Architectures

    The Beauty of Inheritance Architectures

    Spoiler alert: This article is not about inheriting your great-grandparent’s estate. It is rather about how embedding the principles of inheritance into the architecture of your chosen DevOps toolset can yield exponential results. And yes, I realize that you, the customer, have little to do with what a vendor builds its tooling architecture upon. However, you, the customer, can vote with your feet, to search out products that offer features such as inheritance, rewarding vendors that have it and punishing those that do not. The simple question to address here is, Why should the customer care?

    Like anything that can grow, sustainability on scale is a key concern. How much will it cost me when I am maintaining 6,000 applications on this particular DevOps toolset, as opposed to the six applications I started with? You can also infer maintaining a higher number of technologies as having the same need. For example, if I need to maintain three different types of databases, two different middleware, four different hardware platforms on “x” different versions of operating systems … you get the idea. The larger the application portfolio (in sheer numbers of apps), or the more technically complex the portfolio (in sheer numbers of technology components in an app), eventually will drive you to worry about how easy it is to maintain these applications within your chosen DevOps toolset.

    The capabilities of your chosen set of DevOps tooling will govern the costs of both the initial adoption of your services as well as the ongoing maintenance of the application portfolios you serve. It is hard to sell the value of DevOps to the business when it begins to see a one-for-one increase in DevOps engineering staff for every new application or technology venture under consideration. However, one of the chief culprits in driving large DevOps support staff organizations boils down to the lack of an inheritance architecture in the tooling that was originally selected.

    Setting a Common Language

    So before we discuss how inheritance will benefit us, first let’s agree on a common vernacular of what an “application” is. When you say the word “app” to someone, you get a wide variety of responses on what that means—from something that sits on your phone to a cluster of things that sit on the mainframe. Let’s begin by examining what an “app” is by the effort it takes to onboard and then maintain it on our DevOps sets of services (regardless of platform).

    Consider for a moment the hierarchies by which we could describe software constructs. It is relatively easy to discuss how software works in terms of hierarchy. First, at the lowest layer, we could describe having multiple technology components (or the lowest level of a particular piece of technology that must be addressed in a build). Next, assembling these components together forms the picture of what a single application looks like. Finally, assembling multiple applications that have a common symmetry or common purpose together form a system or series of apps.

    An imperfect but quick example could be the Microsoft Org Chart component of its Office application(s). For purposes of example we will argue that Org Chart, Clip Art Manager and Equation Editor are all components of the application Microsoft Excel. To build a new version of the application Excel, we would have to deal with each of these individual components that, once assembled in our build, would create a new version of Excel. When you combine more of the apps—Excel, Word, Outlook and PowerPoint—you get the system, or series, called MS Office. (And, yes, I know this is a less than ideal example, but it is meant only to illustrate a point.)

    Assessing the Work Effort

    The reason for discussing software in this way allows us to begin to discuss the level of effort to automate it. By addressing the work effort to automate each component (by type), we can begin to estimate how long it will take to complete “the automation of software.” For example, if my homegrown application has a source code component, a configuration file component, a database component and a middleware component, then I have four components I must automate (or include in the build and deploy process) to assemble a new version of my application.

    This gives us a “least common denominator” language for assessing how much effort it takes to automate a given app. I have seen most common applications have only two to three components. But I have also seen several apps that, due to their unique architecture and construction, have 30 to 50 components a piece. Obviously, it is easier to automate an application with only three components than one with 50 components. The other factor we need to apply on the ease of automation is whether I have encountered this particular component technology before in my prior automation efforts.

    Let’s say, for example, that the application I am estimating contains Java component(s). If I have encountered Java automation prior to this, I should have constructed a template for Java type component automation that I can use for a situation just like this one. In this situation a familiar technology component is far easier to automate than a new one (or, rather, new to me). However, if this is the first time I have ever seen Java, then it may take me a bit longer to construct the template I want for this app, and for future apps I encounter, that also use this technology type.

    The Beauty of Inheritance

    Here is where inheritance begins to play its critical role. If my DevOps-build construction tooling includes inheritance, then I could construct a “master” template or framework for each type of technology as I encounter it. A master template for SQL databases or Oracle or DB2, as an example. Or, I could make a master template for WebSphere or SharePoint types of technologies—one master for each kind of programming language we have in my shop. You get the idea.

    Having done this allows me to have a limited number of master templates, in which I can maintain all the critical steps and processes my company wants me to do for a given technology type. If different business organizations or development organizations need a variation, they can create a “child” of my master and vary it according to their own needs. As long as the linkage is maintained to my parent/master template, I can make the big, overarching changes driven by corporate mandate, and the business unit can continue to vary items to their unique needs.

    Having inheritance in this instance allows me to roll out changes to an entire organization, such as, for example, a security patch or process, for a given technology by simply adjusting the master template of that component. Inheritance automatically flows my change down to all the “children” of my master and possibly to all their “children,” should the need arise. The alternative is having to go through the specific automation of each application that has the targeted technology component and updating it manually, hoping I don’t miss one, and hoping the steps I may need to adjust do not break the old automation instead of alter it the way I needed.

    Imagine having a WebSphere subject matter expert able to adjust every client application that uses your corporate installation of WebSphere by simply adjusting one master template. Or having a network guru, or database expert, be able to do the same for all applications that use their particular technology component all at once. Inheritance preserves the real-world need of having limited alterations of the same technology type within the same company, flowing down changes from parent to child.

    Having a one-size-fits-all is simply not practical. Using copy/paste requires repeating updates to each impacted template or framework manually. But having the ability to create offshoots from a main, and not lose the connection to the main, allows corporatewide alteration from a central point of control. The costs of maintenance or administration are formidable; inheritance can reduce them. This allows a much higher ratio of applications or technology components supported by a much lower number of engineering staff.

    A Wider Impact for Inheritance

    Inheritance is keenly useful in release orchestration software as well. Most vendors offer the cut-and-paste approach: you copy the last the release you did, alter it slightly and use it again. That might work for some organizations. But having an inheritance feature creates entirely new dynamics on how a release is constructed for a given line of business and how corporate governance can be implemented with minimal effort. In this case, a corporate master release template can be constructed, and child releases can inherit from the master all the main governance required without risk of omission.

    I realize inheritance architectures might not be at the top of your list when evaluating DevOps tooling solutions. But I would submit it is a strategist’s job (if you need one, let me know, I am looking 🙂 ) to point out how long-term sustainability can be greatly benefited by this architectural feature—even if the vendor would love nothing more than to steer you away from it. If you already have committed to a tooling solution, you may want to bring this up at your next vendor features performance meeting. If not, you should at least consider what having it means to your internal cost structure, and what not having it will add to the true cost of ownership of any proposed solution.

    To continue the conversation, feel free to contact me.

  • Culture Change: Metrics, Tools Make a Difference

    Culture Change: Metrics, Tools Make a Difference

    Change is in the wind.

    This time of the year, as winter weather gives way to springtime renewal, this concept isn’t a cliché but rather a concrete, visceral experience. You go outside, smell the earth defrosting and know for certain that change is happening all around you. You can feel it.

    Such is the case on the current DevOps landscape. Seemingly every conversation, with everyone from end users to industry analysts, reaffirms that people are making significant progress toward greater adoption. For an increasing number of organizations, DevOps is maturing from a set of budding goals into a tangible reality.

    At the center of this massive shift within the software factory is the issue of cultural change; as we know in all things DevOps, culture trumps all else. So how are organizations affecting, empowering and measuring all this seismic transition, in particular the human elements?

    During the recent DevOps Virtual Summit, leading practitioners representing industries ranging from banking to travel and technology shared their views on the progress of cultural reinvention.

    From management tips, to tools and even metrics useful for driving transformation, here are some of their observations:

    1. What’s the most critical cultural aspect of planning?
      Kazi Whitfield, director, Quality Engineering, MUFG Union Bank, N.A.: Scalability on this is the key; if you’re doing this at a new organization, you need to understand how to scale it, and starting at some smaller level helps you get there. It also helps each individual and those components that are changing to truly understand what’s changing, and why it’s changing; I think that’s a huge part of the conversation that has to happen.
    1. What’s the biggest obstacle to cultural change?
      Eric Warheit, senior manager, Environment Management, Southwest Airlines: The challenges that we’re working through are organizational in nature. It goes back to the question of localization, and people maintaining a mindset that they’ve previously had, and what we do to create the conversation around what’s possible and the key impacts that we want to have. Ultimately, it’s about being productive with quality, and creating those common values.
    1. How can tools enable cultural change?
      Eugene “Bear” Lehenbauer, group manager, Information Technology, Citrix: If I have to pick one that links them all, a ring to rule them all, that’s when you have an orchestration platform that can take what individual teams are already using and plug right in, or you can take what needs to be automated and automate it; that orchestration harness can now become that common thing that links them.
    1. How can you ensure cross-culture collaboration?
      Warheit: Creating that concrete example, that proof case tying analytics to it, so that we have real measurable ways to make good business decisions, and being able to foster that perspective so that we can look across the organization, as opposed to in a localized way, is really critical to be able to foment that collaboration that’s needed.
    1. How do you measure cultural change?
      Daniele Romano, product owner, Continuous Delivery Pipeline, ING Bank: The most important thing as an organization is to able to apply an analytical approach, meaning that for each team or department you should look at the current situation and have in mind the target situation, and then have different means to guide everyone toward that target.
    1. What’s a key cultural change metric?
      Lehenbauer: We’ve a developed an agility rating scale that allows people to earn scores for something as basic as change management. Based on how those key metrics behave, and there are several of them, but they are fact-based and they’re user impact driven for the most part, they reward the right behaviors. Those types of things can draw out some of DevOps principles results, but also have hard data behind them.
    1. How do you recognize advocates of cultural change?
      Whitfield: Being able to phase reward and recognition into the process is key. Organizations are doing a lot of great things and making millions of dollars, and as the individuals that are coding or testing are driving it, you have to understand how you get to those people’s hearts, to make them passionate about what they’re doing. That’s something that becomes key and it’s something that’s different per organization.
    1. What’s the best way to gauge successful cultural transformation?
      Romano: There is a really only one critical human metric, which is happiness. So, if engineers are happy and they can deliver fast, at a high quality and with intense focus—that’s a very unique but important metric.

    So there you have it, some of today’s most critical elements in driving the cultural transformation necessary to succeed in the DevOps world.

    If you missed the DevOps Virtual Summit you can still view recordings of all the sessions including this customer panel, and stay tuned to this space for more excerpts and highlights.