devopsy.kinsta.cloud

Category: Best Practices

  • Why You Need a Multi-Cloud and Multi-Region Deployment Strategy

    Why You Need a Multi-Cloud and Multi-Region Deployment Strategy

    As I began writing this article, I encountered some technical challenges in my attempt to provision three cloud providers in three separate regions to simulate multi-cloud and multi-region deployments.

    I couldn’t get one of the regions to behave as I expected when provisioning the environment. As it turned out, the region (and by association, cloud) was down due to a cut undersea cable, and it would take several weeks until the outage was mitigated.

    Fortunately, there are plenty of clouds in the sky to choose from, but this experience emphasized the importance of multi-cloud redundancy, especially in a production environment.

    This article examines the benefits of adopting a multi-region and multi-cloud deployment strategy for better resiliency, regulatory compliance and optimal user experience.

    Resilient Reasons to Choose Multi-Cloud and Multi-Region Architectures

    The first reason to choose a multi-cloud or multi-region environment is for data resilience. When our data is in a single region or on a single cluster, there is a good chance that you will experience data failure. You need a mitigation strategy if you want to keep the data lights on 24/7. These strategies fall into two buckets: Multi-region and multi-cloud.

    We first need to define the differences between multi-cloud and multi-region. The architectures are simply the implementation details of how we plan for one versus the other. However, depending on your database provider, those details can represent a significant time investment, which we’ll look at later.

    Multi-Region

    With multi-region, we typically deploy our applications to globally distributed regions. However, regional deploys are subjected to physical faults. Inclement weather, political unrest, acts of foreign or domestic terrorism and other physical variables have all led to regional outages in the past.

    Regardless of the cloud, the regions are exposed to similar physical faults. Instead of losing your data, we can switch to another region to handle the workload. Even though much as our industry hates latency, it is always preferred over loss.

    Deploying to multiple regions allows for failover handling. If the Sydney region goes down, Singapore may still be online. It would be a globally bad day if the same physical faults simultaneously impacted Frankfurt, Sydney and Seattle, for example.

    Multi-Cloud

    Physical faults are only one of the problems that public and private clouds encounter. User error, poorly configured networks and even banal errors can have substantial consequences that impact entire clouds, either from the provider or the user.

    In these circumstances, regional failover won’t help you. An expired certificate, accidental deletion or disabling the wrong port will take you offline, regardless of whether you’re in Seattle or Sydney. If that happens, someone somewhere is most definitely awake and is likely very unhappy.

    Choosing a multi-cloud configuration protects you from these types of faults so that if Google, for example, has interruptions, AWS is still online—or IBM, Deutsche Telekom, OVH or Hetzner.

    The Correct Answer: Deploy Both!

    Multi-region is a go-to method to mitigate against mostly physical faults. Multi-node clusters in each of those regions handle general data corruption or failures on a more granular level.

    Multi-cloud is a go-to method to mitigate configuration or software-level faults, which are not that uncommon.

    The correct answer is to blend the two, working within operational cost constraints and general survivable requirements from regulatory and business objective use cases. For example, I do want access to my heart-rate data from two years ago, but the order of browsing across Amazon shopping is less important.

    Regulatory Reasons for Multi-Region Architecture

    Apart from data resiliency, most companies serving a global market are subject to a number of geopolitical influences, as well. Most users will be familiar with the GDPR regulations ensuring European data stays on European soil. Further regulations require the ability for government oversight to request data at will from service providers, which requires its own set of reasonable isolation from those not in those jurisdictions.

    Using a multi-region database provider allows companies to ship single binaries of software without creating isolated applications for different countries—a practice that was the norm not too many years ago.

    Performant Reasons for Multi-Region Architecture

    Co-locating data and access provides the optimal pathway for getting a user’s data to the end destination. It goes without saying, but the less copper your data has to traverse, the faster the experience will be for your users. Retention and regulation aside, in many cases, you need to mitigate latency caused by region separation.

    It’s not just the individual user’s data. Imagine a sales enablement platform where an account executive needs to manage all of APAC or EMEA. If these accounts are optimally stored for reads on geo boundaries, you can remove seconds of delay between page navigations for your account manager. We all know how much account executives like to click!

    Choosing a DB Provider With Multi-Region and Multi-Cloud

    At this point, you should be convinced of the benefits of multi-region and multi-cloud deployments. But at what cost? Creating replication, failover, logging and synchronization mechanisms is a big, expensive ask. Even with deployment tooling, coordinating all the pieces for multi-region deploys is still a significant operational lift, let alone multi-cloud, where variables in APIs come into play.

    Choosing a single provider that deploys across regions and clouds means you get to benefit from their collective experience while reducing a significant burden.

    Having a database provider offer one-click multi-region deploys is the way to go, but you still need to handle the data access layer. The role-based access, federation and other settings that enable your developers still need to be distributed along with the underlying data sources.

    You need a provider that makes it simple to deploy the same configuration for horizontally scaled instances, leaving you to pick your geo router of choice to distribute the request to the nearest point of presence.

    Closing

    Multi-cloud and multi-region deployments provide significant redundancy, scalability and flexibility. Not only does this enable faster access times and improved reliability, but also provides the flexibility required to adapt to disruptions or unpredictable increases in traffic.

    Deploying multi-cloud and multi-region is no small feat–but the peace of mind and security it provides make it worth the effort.

  • How Good Developers Become Good Engineering Managers

    How Good Developers Become Good Engineering Managers

    Being a good engineering manager means that, to some degree, you have a natural affinity for the work involved. And if you’re a good developer, you likely already have some of those innate skills and attributes.

    But simply having the skills doesn’t mean you’ll enjoy being an engineering manager. For example, the nature of the role is that you’re no longer the one writing the code. I’ve known great architects who became great managers, but they ended up going back to being individual contributors (ICs) because they missed working with nitty-gritty coding details and the joy that brought them.

    It helps to take a realistic inventory of your skillset and preferred working style before making the transition. There are certain competencies you should have completely dialed in, and other tips to know that will help your transition.

    Basic Skills of a Good Engineering Manager

    If you’re a good engineer, you likely already have the following foundational skills that also make a good engineering manager. With these in your toolbox, you’ll have a better chance at success.

    1. Attention to detail. The buck stops with the engineering manager. You need to know when to push and ask questions and how to set outcomes for your team’s accountability. If you work loosely as an IC and need reminders about your own work, you probably won’t do well as a manager.
    2. Amazing personal planning skills. As an engineering manager, you’ll be planning an entire team’s work. If you’re terrible at planning your own work, maybe the engineering manager role isn’t for you.
    3. Speak intelligently about broad technical topics. If you’re a good developer, this shouldn’t be a concern. You’ll need this knowledge to advocate for your team and justify their work to upper management.
    4. Team player attitude. Even as an IC, you should have a collaborative, team-oriented mindset. It goes without saying that you’ll need this when leading others.
    5. Interest in managing others. This might seem obvious, but if you don’t have a natural interest in managing others and in the people side of engineering, you’ll be unhappy as an engineering manager.

    Making the Transition From Developer to Engineering Manager

    If you can do all of the above and are ready to make the transition from developer to engineering manager, here’s what to be aware of as you navigate that process. These tips aren’t meant to deter or discourage you but rather provide a helpful dose of reality.

    Satisfaction in your work takes new forms. If you’re an excellent developer, you know the value you bring and likely get personal satisfaction out of the contributions you make. As a manager, it’s harder to see the direct value you provide. Shift your mindset and realize that now, you’re the one enabling developers to do that rewarding work, and that’s the value you provide. You’ll learn to take enjoyment and fulfillment from your team’s accomplishments.

    People take up a lot of your time. What most frequently catches new engineering managers off guard are the people-related aspects of the role. They’re not prepared for how much time managing people takes. I’ve seen it over and over again where a new manager thinks they can still set aside time to code, but you really have to prioritize learning the craft of management.

    Remember to communicate upward. In the absence of information, it’s human nature to assume the worst. When I was at BitBucket, I assumed that because I had always done a good job, the leaders would think my team was crushing it. The opposite was true; leadership thought everything was on fire.

    I started sending updates every two weeks to various levels of management, and that’s when I learned the power of upward communication. I realized I needed to sell my team’s work, contributions and challenges. Learn what higher-level leaders are looking for and then give them that information regularly.

    Context-switching is part of the job. As an IC, constant context switches kill your day. But being an engineering manager means dealing with endless context-switching. Going to multiple meetings is baked into the job, and you’ll still have to find ways to block off time to get other work done.

    Shield your team. You also need to be prepared to help protect your team from context-switching. You might learn information that’s important to your team during your many meetings, but not all of that needs to be communicated with them. Filter out the junk so it doesn’t rain down on them and they can maintain their flow.

    Being an engineering manager can be a rewarding role that has a considerable impact on an organization. The more prepared you are for the realities of the role and its differences from being a developer, the more you’ll enjoy it and the more successful you’ll be.

  • The Now and Next of Automation

    The Now and Next of Automation

    Today’s never-ending flood of data has many of us drowning in noise. And while humans are still essential for thoughtful analysis and driving action, it’s no longer possible for a human to parse gigabytes of data alone.

    How do you begin to filter out the noise and then understand what to do with it? How can you get to the root cause of an issue so you can act on and resolve it? Most of all, how can you do all of this while being more productive and driving more innovation instead of constantly putting out fires?

    Automation is the obvious answer. For many organizations, automation isn’t just about efficiency – it’s about survival. Automation can make organizations more agile, resilient, and capable.

    Where you sit within an organization or team, however, often dictates how automation can help with your specific goals or alleviate your immediate pain points. Mandi Walls, a DevOps advocate with PagerDuty, offers some advice on how even less experienced teams can use automation to perform simple tasks, solve problems and improve the work environment.

    We hear a lot these days about A.I. and machine learning, or tools like Chat GPT which to some is a harbinger of robots finally replacing humans. Putting the hyperbole aside, where do you see automation today? Is it still a “nice to have” for engineering teams or a “must-have” to do their jobs effectively?

    Automation is definitely a “must-have.” Manual processes are risky; they can inadvertently introduce too many errors, especially in complex environments. The scope and complexity of modern technical ecosystems is just too vast for teams to effectively cope with using only manual methods.

    This is especially true for teams that develop and manage customer-facing services. Having a successful product leads to more customers, which requires more capacity and creates more complexity. To deliver the kind of performance customers expect, teams need to be fast and efficient. That’s where automation comes in.

    What are some specific types of automation that engineers and IT pros should be aware of?

    Some automation has become more or less ubiquitous, such as tools that enable configuring and building large numbers of systems with little human intervention. Other processes might feel less like “big-A” automation–for example, continuous integration/continuous delivery (CI/CD) pipelines are automation.

    Humans also aren’t manually building many software packages anymore. The use of automation in the software testing process has greatly expanded in the last several years, changing the jobs of QA engineers significantly.

    Is self-service automation – aka true autonomy that doesn’t require human intervention – the ideal end state for a digitally mature organization? When is the best time to jump in?

    Organizations that wait for a perfect opportunity to automate their processes risk wasting a lot of time and resources in the interim. Teams can look at any of the tasks that they regularly perform and assess them as candidates for automation.

    Tasks can be assessed in a number of different ways, but for teams just starting out with automation, seeking tasks that are low complexity and low impact is a strong starting point. Even if we look just within the realm of tasks that might be utilized during incident response, teams might have any number of tasks related to information gathering, log preservation, system diagnostics, and other telemetry that responders run for every incident. There’s no reason to require humans to run those well-understood tasks!

    As teams get more comfortable with creating and using automation, they can seek out more complex tasks to automate. And it’s entirely possible that there will be tasks that teams aren’t able to automate fully! Those tasks might be too complex, require a lot of human input or interaction or have too many edge cases for the team to create automation for.

    Are the principles of automation the same in simple versus more complex environments like Kubernetes?

    The principles are essentially the same, but the constraints and tools may be different. In some environments, a simple shell script might be enough to accomplish our goals. In other environments, we might need to make use of APIs or vendor SDKs to do what we want.

    On that topic, is self-service automation in cloud environments the preferred path?

    Actually, self-service is sort of how the cloud really got going! Development teams armed with nothing more than a credit card got the environments they needed without going through lengthy procurement processes.

    Now that cloud is a critical part of many workloads, organizations are putting more rigorous guidelines in place. The cloud platforms are also much more complex, with more service offerings than in the early days, which makes automation more important than ever. Automation helps organizations ensure that their cloud environments meet their security and compliance requirements, helps manage spend and still provides enough flexibility for all teams to get work done. And with self-service automation, developers can request an environment in the cloud to develop their service and have that service meet all the organization’s standards without having to keep up with all the details themselves.

    What about toil? Can that be fully eliminated through automation, or is some toil still necessary?

    I’m not sure I’d say toil is necessary, as much as it tends to be inevitable. There will likely always be some boring, repetitive tasks that are too unpredictable or unstable to automate well.

    That said, teams certainly don’t need more toil; having too much repetitive work tends to lead to less job satisfaction and higher attrition rates, so teams should be looking to minimize as much toil as possible where they can.

    Is there one myth or misconception, either about automation overall or self-service automation in particular, that you’d like to clear up? What’s the reality?

    First of all, don’t worry. You won’t be able to replace your team with a set of shell scripts! Seriously though, automation isn’t about “doing more with less.” It’s about doing more with the same. Automation is an approach to help teams cope with the day-to-day running of complex environments and will need to be managed as part of those environments. Managers need to make sure they are allotting time for the creation and maintenance of automation in the same way they would for other tools and applications.

    Interested in learning more about PagerDuty? Stop by Booth S49 at KubeCon to speak with an expert or see a demo!

  • 5 Key Performance Metrics to Track in 2023

    5 Key Performance Metrics to Track in 2023

    It’s never a bad time to adopt healthy new habits for the mind and body, and for developers, it’s always a good time to consider new best practices at work. As 2023 rolls on, I’ve been thinking about some key performance metrics that help to ensure critical applications and websites perform optimally throughout the year—especially during unpredictable spikes.

    To see why this is important, just look at the problems Ticketmaster had in November 2022 when Taylor Swift’s concert tickets went on sale. Fans of the pop idol (including me!) swamped the Ticketmaster site and brought the system to its knees, resulting in frustrated customers and a wave of bad publicity for the company.

    These usage spikes and the problems they can produce are not uncommon. The holiday sales period, starting with Black Friday and Cyber Monday, is notorious for technical issues that can cost retailers millions due to lost sales and other problems. And it’s not just retail. A few years ago, the Internal Revenue Service (IRS) had to extend its tax deadline after a rush of last-minute filers knocked its systems offline.

    Fortunately, developers can mitigate these problems with the right planning and practices in place. There are key performance metrics that developer teams should monitor, especially in times of peak traffic, to avoid application crashes, long wait times—and burnout for the teams who have to fix these problems when they arise.

    Here are the five performance metrics developers should be tracking in 2023—especially on their busiest days of the year.

    Web Vitals

    The largest contentful paint, or LCP, measures the load speed of a webpage. Having a fast LCP is a clear sign that a customer’s experience is optimal as they toggle between pages. In tandem, the cumulative layout shift, or CLS, scores how a user is experiencing unexpected layout shifts. This plays out on busy shopping days: Typically, these pages have ad and sales notifications that affect a brand’s main page layout, potentially causing shoppers to experience unexpected shifts. This can reduce their ability to shop and becomes a hindrance if a developer cannot quickly access and address the issue.

    Error Count Vs. Error Rate

    Error counts typically increase alongside site traffic as more people flock to a website—that’s a natural occurrence. The error rate is more telling because it reveals if a greater proportion of users are experiencing issues with your application or website. Developers who see an increase in this metric should investigate and take action.

    Mobile Monitoring

    More and more consumers are using their mobile devices to shop and manage all aspects of their lives. For example, mobile accounted for 45% of sales during the holiday shopping period in 2022 from October to December. Mobile performance metrics are critical to understanding what is happening on your users’ devices during these busy times. Vitals like frozen and slow frames or cold and warm starts when an app opens provide visibility into how fast views are loading for your users. As became obvious with the Taylor Swift incident, a slow shopping experience can lead to an outcry and a lot of upset customers.

    Outside of vitals related to mobile performance, it’s important to monitor mobile changes through application release health. Developers can see this in the rate of crash-free sessions and crash-free users as traffic to an application increases—which helps to identify abnormalities in the overall health of the app.

    Slow Database and HTTP Ops

    During sustained traffic spikes, database queries and HTTP requests that take too long to execute harm the user experience. Hitting a slow checkout process, or seeing the dreaded spinning beach ball after adding a product to the cart, confuses shoppers. They don’t know if they should refresh the page, and often they will abandon a purchase completely.

    What’s more, if a developer is working on a backend framework, slow database and HTTP ops could also be a sign of an N+1 query problem. At that point, an application is making database queries in a loop and causing performance issues for the purchaser.

    User Misery

    This is a homegrown metric we use at Sentry; tracking user misery helps developers understand a customer’s experience with an application. It is a ratio of unique users who have experienced load times at 4x a configured threshold in a Sentry project and thus serves as a proxy for customer frustration. With this user misery score, our developers can see which transactions have the highest negative impact on users and prioritize fixing them.

    Providing a smooth end-user experience is table stakes for businesses throughout the year, but especially during high-pressure, culturally-driven spending moments. In any of these scenarios, developers behind the scenes often work long and odd hours to address performance issues and ensure the most seamless experience. You can’t prepare for everything that can possibly go wrong, but you can be equipped with the right tools and metrics to quickly identify and remediate any problems that occur.

  • 5 Unusual Ways to Improve Code Quality

    5 Unusual Ways to Improve Code Quality

    Code quality refers to how well-written and maintainable software code is. It encompasses factors such as readability, efficiency, scalability, reliability, maintainability and security. High code quality ensures that software is easy to understand, modify and extend, has fewer bugs and is less likely to break down. It also reduces development time and costs and improves the overall user experience. One common approach to determine code quality is to use automated tools that analyze the code based on predefined quality standards and generate a report with a numerical score.

    Why is Code Quality Important?

    Generating high-quality code ultimately enhances the end user’s experience. Focusing on creating quality code can also help consumers understand the safety and reliability of the software or program they’re using.

    Low-quality code can affect user experience and satisfaction, making it critical to write high-quality code that functions properly. When developers write low-quality code, it can create vulnerabilities that expose the software to cybersecurity threats, resulting in bugs and functionality issues and impacting the software’s reliability. 

    Here are some additional reasons high-quality code is important: 

    • Enhances code readability: Readability is important because it leads to higher rates of understanding among other developers. A code that’s easy to read is also easy to improve.
    • Improves the level of program sustainability: Program sustainability measures how long a code can survive with minimal changes. High-quality code can last for years, making it more sustainable.
    • Increases transferability: Transferring code from one platform to another is a complicated process. It’s much easier for developers to transfer high-quality code, as they only need to make minimal changes.
    • Lowers costs related to program or software: When software lasts for a long time with few errors, it can save a company money. Instead of paying for additional time to rewrite low-quality code, the company can rely on the original, high-quality source code.

    5 Unusual Ways to Improve Code Quality

    Here are a few ways you may not have thought of which can help your development team improve code quality and end-user satisfaction.

    Bug Bounty

    A bug bounty program is a type of crowdsourced security testing in which an organization offers rewards to independent security researchers or ethical hackers for identifying and reporting vulnerabilities in their software or systems. By offering rewards for vulnerabilities, organizations can incentivize security researchers to find and report security issues before they can be exploited by attackers.

    Bug bounty programs can improve code quality in several ways:

    • Incentivize security testing: Bug bounty programs incentivize security researchers to test an organization’s software or systems for vulnerabilities. This can increase the amount and quality of security testing that is done, leading to a more secure product.
    • Encourage responsible disclosure: Bug bounty programs encourage security researchers to report vulnerabilities directly to the organization rather than publishing them publicly or selling them on the black market. This allows organizations to fix vulnerabilities before they are exploited by attackers.
    • Provide feedback on code quality: Security researchers who participate in bug bounty programs can provide feedback on code quality which can help organizations to improve their development practices and write more secure code in the future.

    Auto-Generated Documentation

    Auto-generated documentation is a type of documentation that is generated from code comments and metadata as part of an automated development pipeline. It is typically generated using tools such as Javadoc, Doxygen, or Sphinx, which parse code comments and metadata to generate HTML, PDF, or other formats of documentation.

    Auto-generated documentation can improve code quality in several ways:

    • Improved code readability: Auto-generated documentation can provide clear and concise descriptions of code functionality, making it easier for developers to understand the code and its purpose.
    • Better code organization: Auto-generated documentation can help developers organize their code by providing a clear hierarchy of classes, methods, and functions, making it easier to navigate and maintain the codebase.
    • Consistency: Auto-generated documentation can ensure that documentation is consistent across the codebase, reducing the risk of errors and misunderstandings.
    • Quality assurance: Auto-generated documentation can be used as a quality assurance tool, helping developers to identify errors and omissions in their code comments and metadata.

    Coding Conventions

    Coding conventions are a set of guidelines and standards that outline how code should be written and formatted to promote consistency and readability. They typically cover aspects such as naming conventions, code structure, formatting, and documentation. 

    By following coding conventions, developers can improve code quality and make their code easier to read and understand, which makes it easier to maintain and modify. They also help ensure that code is consistent across a project or organization, which can make collaboration and integration of different code segments easier.

    Typical elements that coding conventions dictate include:

    • Naming conventions for variables, functions, and classes.
    • Indentation and formatting of code blocks.
    • Use of whitespace and comments to improve readability.
    • Guidelines for handling exceptions, errors, and input/output.

    Quality Gates

    Quality gates are a set of checkpoints or criteria that software must meet before it can progress to the next stage of development. They are typically used in software development processes to ensure that the software is of a certain quality and is ready for release.

    Quality gates can be automated or manual, and they are typically implemented at key points in the development process, such as after unit testing, integration testing, or acceptance testing. They can include metrics such as code coverage, code quality, functionality, performance and security.

    By enforcing quality gates, development teams can catch and address issues earlier in the development process, which can reduce rework, save time and ensure that the final software product meets the required quality standards.

    Smart Code Snippets

    Smart code snippets, also known as intelligent code snippets or live templates, are pre-written code fragments that can be inserted into a code file with placeholders for variables or user-defined inputs. They are designed to help developers write code more efficiently and accurately by reducing the amount of repetitive or boilerplate code they need to write and by providing contextual awareness and intelligence.

    Regular code snippets are static pieces of code that can be inserted into a file as needed. They typically contain pre-written code blocks or templates that can be customized to fit the specific use case.

    In contrast, smart code snippets are dynamic and can adapt to the context of the code being written. For example, a smart code snippet might include placeholders for variables that need to be defined or automatically fill in common code constructs based on the context of the code. 

    Conclusion

    The five lesser-known ways to improve code quality discussed in this article offer a starting point for teams looking to take a fresh perspective on their development process. Improving code quality is critical for the success of any software development project. While there are many established methods and best practices for improving code quality, there are also some less conventional approaches that can be surprisingly effective. By adopting a creative and open-minded approach to software development, teams can identify new ways to improve code quality and build more reliable, efficient and secure software.

    Image Source 

  • DevOps Done Right: How to Succeed in DevOps From Day One

    DevOps Done Right: How to Succeed in DevOps From Day One

    Whether you are diving into DevOps for the first time or trying to do it right this time, DevOps Onramp provides all the information you need for a successful DevOps journey. Join us on May 4, 2023, to learn how to get started with DevOps—and how to do it right.

    You’ll learn:

    • DevOps best practices, processes and the tools you’ll need
    • Tips and tricks from experienced practitioners sharing their real-world experiences
    • The basics of DevOps
    • How to choose the right technologies
    • Common challenges and how to overcome them
    • The benefits of DevOps as a methodology and a career
    • And more!

    As organizations face increasing pressure to adapt faster and become more agile, getting DevOps right and adopting a DevOps mindset is crucial to deliver high-quality software faster and increase business value. And that opens up great opportunities for anyone looking to start a new career or pivot from an existing role in technology.

    DevOps has become the standard of software development methodologies and it’s critical for successful digital transformation. Adopting DevOps—and doing it the right way—can benefit not only your organization but also your customers and end users by fostering innovation, reducing time to market, increasing development productivity, improving product quality and reliability and reducing costs.

    That’s why many big companies, such as Amazon, Netflix, Facebook and many others have adopted a DevOps mindset—and DevOps practices. And as more businesses are looking to move to the cloud and transition into DevOps, there are plenty of job opportunities available for DevOps engineers at various levels.

    DevOps is a growing field with a lot of opportunities for professional growth. According to a research study by Global Market Insights Inc., the DevOps market is expected to surpass $30 billion by 2028. Due to the high demand for skilled DevOps engineers and the shortage of qualified DevOps professionals in the industry, there is no better time to upskill than now.

    Whether you are diving into DevOps for the first time or trying to do it right this time, DevOps Onramp provides all the information you need for a successful DevOps journey.

    At DevOps Onramp Virtual Summit, taking place on May 4, 2023, you’ll get all the knowledge you need to get started with DevOps —and how to do it right. Join us to learn about DevOps best practices, processes and tools and get tips and tricks from experienced practitioners sharing their real-world experiences. We’ll go through the basics of DevOps, how to choose the right technologies, common challenges and the benefits of DevOps.

    One thing is for sure: The future of DevOps is very promising. The rush has started—don’t be left behind!

    DevOps Onramp Virtual Summit - Submit your talk

    If you’re interested in becoming a sponsor, please contact us. For more information, please visit the DevOps Onramp website.

  • State of DevOps Report Finds a Rise in Platform Engineering

    State of DevOps Report Finds a Rise in Platform Engineering

    The speed of software development velocity is increasing, and to enable this, platform engineering is emerging to evolve the DevOps practice. Interest in platform engineering is rising—so much so that Puppet emphasized platform engineering in its 2023 State of DevOps Report.

    A whopping 94% of respondents agreed that platform engineering is helping their organizations better realize the benefits of DevOps. The move toward platform engineering has increased over time, bringing benefits such as increased release velocity, reliability and efficiency.

    Below, we’ll highlight some of the key takeaways from the report. We’ll define what platform engineering is and highlight its benefits. We’ll also consider obstacles on the path to embracing a platform-based development culture.

    Platform Engineering Supplants DevOps

    DevOps has been in use for a decade or more at most institutions, maturing steadily with new automation and infrastructure abstraction. The latest evolution of DevOps is a practice known as platform engineering—51% of organizations have already adopted this practice within the last three years and 93% said it’s a step in the right direction.

    Platform engineering is the discipline of designing and building self-service capabilities to minimize cognitive load for developers and to enable fast-flow software delivery.

    In essence, platform engineering is the process of creating standardized, self-service infrastructure that helps developers get up and running quickly. These self-service platforms typically handle capabilities like deployment (43%), operation of infrastructure (43%), monitoring production applications (42%) and security and compliance (42%).

    These shared internal platforms are maintained by a central platform team that embraces a product mindset to iterate the technology based on user feedback.

    State of Platform Engineering

    Platform engineering is relatively new— 27% of platform teams have formed within the last two to three years. Notably, big tech companies hopped on the bandwagon years ago, and non-tech companies are now following. Regardless, 70% believed their timing was “just right.”

    The goals of a platform engineering team are numerous. According to the Puppet report, 54% said problem-solving is the top goal. This is followed by educating and empowering developer teams (47%), setting best practices (46%) and optimizing iteration (44%). A platform engineering team’s scope of work is primarily focused on automating workflows and building and managing infrastructure.

    The ‘correct’ number of internal platforms is a moving target. The majority of organizations (57%) have between two and four platforms, and 30% have five or more internal self-service platforms. This demonstrates that although platform engineering is somewhat centralized, it’s also flexible to the needs of an organization.

    Benefits and Hurdles

    So, what are some benefits as opposed to traditional DevOps? Well, the most common benefit is improved system reliability, at 60%. This tracks, as a standardized platform could reduce human error and missteps in the development workflow. Following this benefit is improving efficiency (59%), speeding up delivery time (58%) and improving workflow and process standards (57%).

    The most common driver of adoption is increasing the speed of delivery. Slight more than half of respondents (51%) said development speed has somewhat increased since the inception of the platform team, and 42% say it’s increased by “a great deal.” Other top drivers include the need to scale, free up engineering time, control infrastructure spending and avoid duplication across teams. Those with more mature adoption tend to be more satisfied with the result, indicating that results increased over time.

    Of course, no new technological introduction comes without its hurdles, and platform engineering is not immune from certain pain points. For example, 34% said that cycle time is slower than expected. This might be due to nascent industry adoption and a lack of best practices.

    Just under one-third (32%) also noticed resistance to platform team adoption, and 32% reported a lack of communication around changes. Organizations will likely have to increase transparency and reduce siloing to encourage productive platform adoption that satisfies the needs of various stakeholders.

    According to the report, product management skills can help realize these goals. Some of the most fundamental product management skills here include strong communication skills (61%), problem-solving expertise (60%) and the ability to foster collaboration (54%). Many companies will also have to convince senior management to invest in platform engineering—nearly half agreed that senior management doesn’t yet understand its value.

    Future Outlook

    Confidence around platform engineering is increasing, especially for teams with mature DevOps approaches. As a result, the practice will likely gain more traction in the coming year—71% of respondents reported their firm plans to hire people with platform engineering experience over the next 12 months.

    Over the next year, organizations that adopt the practice will want to increase awareness of platform capabilities and set expectations across the organization. This will also include setting governance policies and satisfying the needs of specific developers, departments and the organization as a whole.

    Above, we’ve scratched the surface of the current state of platform engineering, its benefits and its pain points. For plenty of other insights, you can pick up a copy of The State of DevOps Report 2023: Platform Engineering Edition from Puppet by Perforce here.

  • 5 Best Practices for Implementing Cloud-Native DevOps

    5 Best Practices for Implementing Cloud-Native DevOps

    As the world transitions to cloud-native offerings as an industry norm, DevOps is gaining traction for its critical role in supporting more efficient IT infrastructure. DevOps is designed to boost collaboration and communication by streamlining the automation process to expedite the creation and deployment of applications. Implementing cloud-native DevOps requires organizations to make a massive cultural shift. Without this change, businesses are unlikely to be as competitive or fully utilize the innovations the cloud has to offer. 

    There’s always a risk in adopting new technology. Following are five best practices to accelerate your company’s implementation process and address potential challenges head-on.

    1. Think cloud, adopt the cloud.

    Optimize technology with a variety of tools tailored for the cloud. Understanding how technology works in the open source environment that cloud-native DevOps relies on is as essential to successfully managing the components as it is to selecting the right tool for you. Don’t get locked into using one tool. Instead, select the right tool for each task while remaining as cloud-agnostic as possible.  

    2. Implement a dynamic security strategy.

    Not considering security is one of the most common pitfalls organizations encounter when using emerging technology. Security must be built into cloud-native development workflows and thought processes and diligently monitored. If funds are available, appoint a chief security officer responsible for scrutinizing DevOps security in the cloud and creating playbooks for each alert triggered. Cybersecurity is becoming more complex due to the unpredictable nature of adversaries and the sheer volume of technologies involved in each solution. One way to start is with a bug bounty program that can bring hackers to your side of the court.

    3. Commit to ongoing training.

    Perform a self-audit and/or penetration testing to reveal weaknesses in company technology, procedures and people. Invest in training to effectively address any gaps in cloud skills. Providing development opportunities will also improve employee engagement and retention, which is a challenge for many employers. Make sure your employees give ongoing feedback to the training’s content, its maintenance and evolution.

    While it may be tempting only to emphasize new-hire training, the rapidly changing technology landscape requires training efforts be geared toward ongoing learning for the entire team. This investment will pay dividends, with DevOps and the cloud netting major long-term savings. To make training more efficient, it can be conducted remotely. Integrate and target training based on employees’ job functions,  but don’t forget about non-IT employees—they must also have a basic understanding of cloud and cloud-native.

    4. Once, twice, automate!

    Automation is critical to minimize software development life cycle friction and eliminate dependence on manual labor. Provisioning networks, infrastructure, application deployment and setup can be automated and versioned with infrastructure-as-code (IaC) tools (such as Terraform). There are also endless automation opportunities for containers, cloud computing, security, testing, monitoring and more. For instance, containers allow developers to compartmentalize applications to work on components without worrying about their potential impact in other areas–making it easier to construct consistent, tangle-free deployments. However, it may be challenging to manage multiple versions, requiring infrastructure and testing investment.

    5. Performance-test your tests.

    Automated testing will save developers time and reduce human errors. It’ll also allow developers to take advantage of doubling up on testing, making sure every change is stable and benchmarked. Test during the development stage and lay the groundwork for an effective CI process, especially if you roll out patches regularly. Automated testing is easier in a cloud-based system and isn’t dependent on sufficient hardware to make it run properly, allowing cloud hosts to scale up or down easily.

    Final Takeaways

    Cloud-native DevOps is revolutionizing the way applications are being built and managed, and it starts with how we perceive our challenges. Embracing DevOps culture enables organizations to leverage cloud and cloud-native solutions to create reliable services that are easily scalable. While it may look resource-intensive, cloud-native DevOps is becoming a business necessity.

  • Shift Left Testing in Microservices Environments

    Shift Left Testing in Microservices Environments

    By now, it’s common knowledge that the later a bug is detected in the software development life cycle (SDLC), the longer it takes and the more expensive it is to fix that bug.

    In 2017, the Ponemon Institute found that it cost around $80 on average to fix a defect detected early in the SDLC and around $7,600 to fix the same defect if it was detected in the production stage.

    Relative Cost of Fixing a Defect Based on Development Stage

    Design

    Implementation

    Testing

    Maintenance

    Cost of Fixing Defects

    1x

    6.5x

    15x

    100x

    While these numbers can be debated, it’s clear that early detection of defects can save organizations significant time and money while enabling those organizations to deliver higher quality software more rapidly. For these reasons, many organizations are implementing shift left testing practices, including in microservices environments.

    Shift left is a set of practices intended to find and prevent defects early in the software delivery process instead of during the testing and maintenance stages. The idea behind shift left testing is to improve software quality by making actionable test signals and feedback available as early in the SDLC as possible.

    Here, we’ll highlight how shift left testing, in combination with test automation, can better enable both innovation and fast delivery of high-quality software. We’ll also address some of the challenges involved in implementing shift left testing in microservices environments, as well as introduce a valuable strategy for successful microservices implementation.

    Shift Left Testing Benefits

    The waterfall model of development involved the explicit passing of responsibilities between highly specialized design, development, QA, and release teams. It also involved lengthy feedback loops.

    Scrum and agile methodologies made the entire SDLC more flexible and nimble by introducing sprints and allowing more frequent iterative development and delivery. Further, DevOps and DevSecOps focus on removing the silos between development, operations, and security through tooling and automation. As a result, the time to market and quality have improved dramatically. Adding shift left testing into the mix better positions teams to handle the broad range of responsibilities from the design stage through the maintenance stage as effectively as possible.

    Shift left testing focuses on prevention rather than detection. Shift left benefits include the following:

    • Increase efficiency by eliminating bugs earlier in the SDLC.
    • Reduce human errors and associated costs.
    • Increase delivery speed and reduce the time between releases.
    • Improve the quality of software.
    • Gain a competitive advantage.

    It may be helpful to think of shift left testing like this: Test early and in parallel with development, instead of the traditional waterfall model of development followed by QA testing. 

    Shift Left Testing Challenges in a Microservices Environment

    By structuring an application as a collection of loosely coupled, independently deployable services, a microservices architecture enables frequent delivery of large, complex applications.

    While shift left testing practices bring many benefits, shift left testing also comes with three challenges that organizations must address when implementing it across a decentralized, microservices-based stack:

    1. Building and managing each microservice separately reduces development time. However, implementing shift left testing at the microservices level means shift left practices must be applied to each individual microservice.
    2. Microservices developers can choose the programming language and service dependencies best suited to their development needs. However, as a result, shift left testing practices need to be implemented and managed across different programming languages and dependencies.
    3. To ensure that microservices function as expected, teams usually run integration tests, end-to-end tests, and performance tests that assess functionality in the context of the larger stack. Shift left testing is especially challenging in the microservices context because of the complexity of testing against the stack.

    Microservices, each with their own SDLC pipeline

    Pipeline Scaling Considerations

    Several classes of automated shift left tests can be integrated into the DevOps pipeline, including unit tests, integration tests, performance tests, and security tests. Different microservices will have different testing needs.

    The key is to apply shift left testing practices to each microservice while maintaining the flexibility and decentralization of those microservices. The ideal workflow system and DevOps pipeline implementation must simplify pipeline creation while enabling development teams to manage the pipeline and requiring minimal input from DevOps and platform teams. In other words, shift left practices must be tailored to the specific needs of the microservice in question to make the testing process scalable and easy for development teams to manage on a continuous basis.

    As just one example, GitHub Actions and similar CI/CD pipeline tools enable development teams to use GitOps-based practices to configure, manage, and administer the entire pipeline for their microservice while leaving infrastructure testing and testing that relies on or affects other parts of the system to DevOps teams.

    Test Environment Considerations

    Test environments are environments where unit tests, integration tests, performance tests and security tests can be run on an isolated collection of microservices and configurations. The nature and quality of these test environments directly correlate with the effectiveness of shift left testing.

    A cloned production or staging environment can be used for testing when the number of microservices is low and overall complexity is relatively low. However, once an organization reaches a threshold number of microservices or a threshold level of complexity, runaway infrastructure costs and operational burdens make such environments untenable.

    In response, some organizations rely upon smoke testing (minimal viable stability testing) and mock-based integration testing early in the SDLC. While such tests can prove useful in the near term, deferring essential testing to a later stage in the SDLC often introduces test signal and validity problems in the staging environment and can lead to delivery delays and serious issues with software quality.

    So how can shift left testing be used successfully in a microservices environment? The solution involves the use of scalable, ephemeral test environments.

    Ephemeral Shift Left Testing Environments

    The availability of high-quality test environments is a common bottleneck for organizations looking to scale shift left testing practices. Using a few high-fidelity staging environments for signal testing can slow the release process. More to the point, it tends to slow down the entire SDLC and increase stack instability over time.

    Ephemeral test environments offer a fundamentally different approach. These sandbox environments use isolated environments that enable users to test independently without impacting other users. In particular, they enable isolation while sharing resources (e.g., external cloud dependencies and hosted databases) using application-level multitenancy instead of the resource duplication approach in traditional environments.

    The idea is that microservice test environments can be used to test changes against a much larger stack of services. Because multiple tests can be conducted at the same time, test coverage and resource efficiency vastly increase.

    High-quality sandbox environments can spin up in seconds and scale to tens of thousands of microservices. In fact, each pull request, branch, or commit can have its own test environment without significantly increasing infrastructure and operational costs. Any change, no matter how small, can be tested in isolation to ensure performance and stability against the entire stack.

    Shift Left Testing Solutions for Microservices

    A microservices-based organization looking to scale shift left testing across the organization must first address several challenges. Considerations include the need to apply testing practices to separate microservices, the need to accommodate different programming languages and service dependencies and the need to test against the stack.

    The right tooling and automation are needed before shift left testing practices are scaled across the organization. In addition, a cultural shift may be required in order to help developers understand the ways in which shift left testing can expedite and improve the core functionality that they are developing.

    To apply valuable shift left testing practices in a microservices environment, organizations should consider using ephemeral sandbox environments. Using the right set of tools in a sandbox environment can enable development teams to test microservices throughout the SDLC. In doing so, organizations can ensure those services follow security best practices and perform exactly as intended, both independently and against the stack–all while reducing costs, speeding time to market and improving software quality and user satisfaction.

  • 5 Tips to Consider When Introducing Infrastructure Automation

    5 Tips to Consider When Introducing Infrastructure Automation

    Although we discuss infrastructure automation often, the fact is that a lot of manual processes still exist throughout DevOps. Whether it’s rewriting scripts to spin up new servers, updating cloud-native policies or configuring computing environments, manual toil is an all-too-common reality. To make matters worse, engineers commonly create lengthy scripts that work in isolation but are too bulky to be reusable by other developers working with slightly different constraints.

    More infrastructure automation is the key to increasing deployment agility, but setting aside time to automate your daily workflow can easily take a back burner, especially when developers are already behind on their long list of to-dos. So, for the time-pressed, where should they start? What exactly should be automated? And how can we effectively reuse and share automation to help other areas of the business?

    I recently connected with John Laffey, technical product marketing director, Puppet. Laffey has thirty years’ worth of tech support and engineering operations knowledge under his belt and was “doing DevOps before it was called DevOps.” He shared with me five helpful tips to think about when introducing more infrastructure automation into your workflows. Below, we’ll expand on each of those points and consider other best practices around automation as the industry moves toward platform engineering.

    1. Automate Things That Interrupt Your Day

    First, Laffey recommended starting with the things that are easy to fix. These are the tedious actions that interrupt your day. Perhaps a disk runs out of space and troubleshooting takes 15 or 20 minutes. Minor interruptions like this might seem like part of the job, but they definitely add up over time.

    2. Automate Things You Can Reuse

    Secondly, try not to automate things for an ultra-specific purpose, said Laffey. Instead, focus on building automation that is more generic and can have more value to others. This follows the DevOps mantra of teamwork and collaboration. One-off automation that is too verbose or specific to a certain condition won’t apply well across groups.

    3. Automate Stuff That Doesn’t Change Much

    Next, automate the infrastructure that isn’t prone to change. These are fairly reliable things that you’re sure will be part of your systems for the foreseeable future. Because if a component does end up having to be updated often, supporting the automation could be more hassle than it’s worth. “Don’t trade fixing problems for fixing automation,” said Laffey.

    4. Start Small

    Here’s some sage advice applicable to most new projects with a big scope—start small. Laffey recommended starting by automating the easy stuff that has proven immediate value. These tasks should be modular and do one specific thing, like rebooting a machine or draining a log file. “Automate things in chunks that you can put into a toolchain.”

    Too often, teams try to automate everything all at once. But by taking big bites, you can quickly get off track and potentially add to technical debt. If you keep things modular, you’ll find it easier to change and reuse this automation.

    5. Share and Be Open

    This last tidbit is more of a cultural change—don’t code so that you’re the only one who knows about it! Sometimes, engineers don’t necessarily want the world to know about the automation they create. They might worry about automating their job away. Yet, Laffey said, these worries are misplaced. There will always be more work to do, and automating toil away can free up time to help you focus on more exciting projects.

    So, Laffey encouraged DevOps and SREs to be open about the automation they create. Ensure they’re well documented and stored in a centralized repository to allow other departments to reuse your code. This can avoid the duplication of having five different versions of the same script written in different languages within the same organization!

    Next Step: Move From Scripts to Industry-Standard Tooling

    It can be challenging to argue for investing time and energy into infrastructure automation, especially when engineers are hard-pressed to keep the lights on. Yet, small gains in automation can make huge waves in the long term.

    Eventually, said Laffey, you’ll want to move away from using scripts within your toolchain, which he describes as a low-end DevOps maturity. But as you update your process, you’ll want to replace these scripts with more modern industry-standard tools like Terraform or Puppet’s DSL. He recommended slowly revamping scripts in a toolchain as you update them one by one. “Start where you are and avoid a monolithic approach.”

    In essence, platform engineering, Laffey explained, is wrapping DevOps principles at a higher level to make them more efficient. And we will likely see an increased emphasis on infrastructure automation as platform engineering gains more mindshare. “It’s going to become a necessity.”