devopsy.kinsta.cloud

Tag: best practices

  • Automation in DevOps: Advancements, Platforms and Examples

    Automation in DevOps: Advancements, Platforms and Examples

    Automation is the backbone of DevOps. By facilitating seamless integration and continuous delivery, it revolutionizes the ways organizations deliver software, products and services. When applied correctly, automation can significantly enhance productivity, reduce time-to-market, improve software quality and minimize the risk of human error.

    A range of automation tools support these goals, each tailored to a specific aspect of the software development life cycle (SDLC).

    From code integration to testing and deployment, these tools optimize processes and enable the rapid and reliable release of software. In this article, we will explore the role of automation in the DevOps landscape.

    The Importance of Automation in DevOps

    Automation in DevOps is the process of designing, developing and deploying software and systems with minimal human intervention. The goal is to streamline the software development life cycle from design, development and testing to deployment and management.

    Automation provides several benefits:

    1. Speed and efficiency: Automation enables quick development and deployment of software, which results in faster delivery of features and fixes to customers.
    2. Reliability: Automated systems are less prone to human error, resulting in more reliable code deployment and management.
    3. Consistency: Automation guarantees that routine tasks are performed the same way every time, ensuring consistent results.
    4. Reduced cost: Automation eliminates the need for manual intervention in routine tasks, leading to a reduction in operating costs.
    5. Increased productivity: By taking over routine tasks, automation frees up the team to focus on more critical, high-level tasks.

    Automation in DevOps: An Overview

    Automation in DevOps encapsulates various stages, from development and testing to integration, deployment, and monitoring. The process is often referred to as a DevOps pipeline, a set of practices aimed at reducing development times and improving software quality.

    ● Development: Automation begins at the development phase with tools for source control (like Git), which help developers manage and track code changes.
    ● Integration: The next step is continuous integration, where tools like Jenkins automate the build process, allowing teams to identify issues early in the development life cycle.
    ● Testing: Automated testing tools (like Selenium) streamline the testing process, ensuring that the code is error-free and ready for deployment.
    ● Deployment: Continuous deployment tools (like Kubernetes) automate the deployment process, providing fast, consistent and repeatable deployments.
    ● Documenting: Technical documentation is automated with tools like Scriversi to ensure updated, error-free documentation through each version and release.
    ● Monitoring: Finally, automated monitoring tools (like Prometheus) offer real-time insights into the application’s performance, facilitating proactive troubleshooting.

    Automation’s main objective is to minimize manual intervention, thus reducing the risk of errors, improving efficiency and fostering a culture of shared responsibility.

    Examples of Automation in DevOps

    1. Continuous Integration/Continuous Deployment (CI/CD): CI/CD is a critical element of DevOps automation. With CI, developers merge their changes back to the main branch as often as possible. The CI/CD server automatically builds and tests every change. For example, Jenkins, a popular open source automation server, allows developers to automate various stages of their build pipeline.

    2. Infrastructure-as-Code (IaC): This practice involves managing and provisioning computing infrastructure with machine-readable script files rather than physical hardware configuration or interactive configuration tools. For instance, Terraform, an open source IaC software tool, provides a consistent CLI workflow to manage hundreds of cloud services.

    3. Configuration Management: This is the process of maintaining a system’s configuration in a known, consistent state. For example, Ansible, a configuration management tool, helps to automate software provisioning, configuration management and application deployment.

    4. Monitoring and Logging: Automated monitoring and logging systems gather data on how software and hardware are performing and alert the team to any problems. One example is Prometheus, a popular open source monitoring and alerting toolkit.

    5. Technical Documentation: Proper documentation is crucial in the software development process. Yet, it is often overlooked or not updated frequently due to time constraints or the tedious nature of the work. When automated, it simplifies and automates the process of creating and maintaining technical documents.

    Final Thoughts

    In the world of DevOps, automation is not a luxury but a necessity. By automating repetitive and error-prone tasks, teams can focus on what they do best: creating innovative solutions that meet customer needs. From CI/CD, IaC and configuration management to monitoring and logging, automation simplifies the software development process, reduces human error and enhances speed and efficiency. Tools like Jenkins, Terraform, Ansible, Prometheus and Scriversi support this trend, providing various automation capabilities that make the DevOps approach more efficient and effective.

  • GitLab Survey Reveals DevSecOps Gains

    GitLab Survey Reveals DevSecOps Gains

    A global survey of 5,010 IT leaders, CISOs and developers published by GitLab this week found well over half (56%) of respondents are working for organizations that are using DevOps or DevSecOps practices. However, 72% of respondents said they are using a DevSecOps platform (36%) or are considering adopting one in the next year (36%).

    Overall, less than a third of survey respondents (30%) said they were “completely” responsible for application security, with 53% reporting they are part of a larger application security team. Well over a third (38%) of the 1,453 security professionals surveyed said they are part of a cross-functional team focused on security.

    License compliance checks and security capabilities for cloud-native or serverless tied for the top of the list of current priorities (19%), but shifting security left (29%) was the top focus for the coming year. Nearly three-quarters (74%) of security professionals said their organizations either shifted left or plan to in the next three years. Nearly as many (71%) said a quarter or more of all security vulnerabilities were discovered by developers.

    Top frustrations identified specifically by security professionals included testing happening too late in the development cycle (43%) and difficulty prioritizing vulnerability remediation (41%).

    Those frustrations might increase in the coming year, with 85% of security professionals reporting they either have the same or lower budget in 2023.

    On the plus side, nearly two-thirds of the 1,954 developers surveyed said they are either using artificial intelligence (AI) and machine learning (ML) algorithms in testing today or will be in the next three years. A full 62% said they used AI/ML to check code, while 53% use bots for testing and 36% use AI/ML for code review.

    Nevertheless, more than two-thirds of security professionals (67%) said they are concerned about the impact of AI/ML capabilities on their job, with 28% of them admitting they are “very” or “extremely” concerned. Of those respondents who expressed concern, 25% said they are worried about the potential for AI/ML to introduce errors that will make their job more difficult.

    David DeSanto, chief product officer at GitLab, said GitLab is also working to address those concerns by embedding a range of AI/ML capabilities into its platform to optimize DevSecOps workflows and, most recently, enhanced its GitLab Remote Development module to make it easier to centralize the maintenance and security of development environments.

    GitLab this week also announced it is partnering with Oracle to make it easier to use the GitLab continuous integration/continuous delivery (CI/CD) platform to run AI and ML workloads on the Oracle cloud platforms. AI/ML is going to play a critical role in enabling organizations to build and deploy secure code without slowing down the pace of application development, said DeSanto. In fact, many organizations will soon be consolidating toolchains to enable them to achieve that goal, he added.

    The GitLab survey found 57% of security respondents said they use six or more tools, but many of those tools are legacy software composition analysis (SCA) tools that will be replaced as more responsibility for application security shifts left toward developers, noted DeSanto. Two-thirds of survey respondents (66%) reported they want to consolidate their toolchains this year.

    Collectively, the GitLab survey makes it clear that DevOps and DevSecOps practices and platforms continue to evolve rapidly. The challenge, as always, is keeping up with the pace of that change.

  • Awareness of Software Supply Chain Security Issues Improves

    Awareness of Software Supply Chain Security Issues Improves

    A global survey of 167 software professionals suggested that, while there is a lot more awareness of application security issues, the adoption of DevSecOps best practices is still not pervasive.

    The survey, conducted by Chainguard, the Eclipse Foundation, the Rust Foundation and the Open Source Security Foundation (OpenSSF), found that just over half (51%) of respondents are either concerned or are extremely concerned about software supply chain security.

    In addition, only a third of respondents (33%) reported they are familiar with the Supply-chain Levels for Software Artifacts (SLSA) framework for building more secure applications. Well over a third (36%), however, said they had never heard of it.

    Only a quarter of respondents (25%) reported that their team always signed built artifacts, the survey found.

    The survey identified the most challenging issues developers are encountering and found they are reproducing builds (64%) followed by creating hermetic builds that are isolated from other software libraries (62%).

    Mikaël Barbero, head of cybersecurity for the Eclipse Foundation, said while awareness of software supply chain issues is higher, it’s apparent that other priorities are still getting in the way of adoption of DevSecOps best practices. For example, software bills of materials (SBOMs) may generate a lot of headlines, but their adoption remains mixed, he added.

    John Speed Meyers, a security data scientist for Chainguard, noted that incentives still need to change; too many managers that hire developers still emphasize performance and delivery times over application security. Many developers, as a result, still tend to view cybersecurity as an obstacle to be overcome or worked around rather than a core capability that they will be graded on, he added.

    In theory, as responsibility for application security shifts left toward developers, overall cybersecurity will improve. Most developers, however, have little to no cybersecurity training, so the odds they will be able to resolve these issues are low. Instead, many organizations are either trying to embed cybersecurity professionals within DevOps workflows or, at the very least, deputize at least one developer to advocate for cybersecurity within those teams.

    Regardless of commitment level, it’s apparent that organizations need to address everything from locking down developer environments to thwart phishing attacks to ensuring application builds are not inadvertently infected with malware. The challenge organizations face is both technical and cultural, so the amount of effort required should not be underestimated.

    In the meantime, organizations should expect to see cybercriminals increase their attempts to compromise software supply chains. Infecting malware into software components that will be used by multiple downstream applications is, from their perspective, one of the most efficient means of wreaking havoc. Right now, organizations make it too easy for cybercriminals to compromise application environments.

    Organizations should also recognize that tolerance for insecure applications is also declining, as evidenced by the National Cybersecurity Strategy put forward by the Biden administration. It’s not so much a question of whether organizations that create software will be held liable for application security issues as much as it is to what degree.

  • Strategies To Consider When Adopting Platform Engineering

    Strategies To Consider When Adopting Platform Engineering

    Platform engineering helps centralize the responsibilities associated with maintaining internal infrastructure. And it continues to be an emerging discipline—according to the 2023 State of DevOps Report, 51% of organizations have already adopted platform engineering within the last three years, and 93% said it’s a step in the right direction.

    I recently met with Hope Lynch, senior director of platform at Cloudbees, to gain some insights on implementing platform engineering. According to Lynch, platform engineering has emerged at a critical juncture. Developers are overloaded, burnout is on the rise and tooling sprawl is starting to rear its ugly head. To respond to these trends, platform engineering has emerged to take the heavy load off internal software infrastructure management. But the practice requires keenly balancing responsibility with control and treating the platform as a product.

    Interestingly, Lynch believes that many IT departments—whether they know it or not—are already implementing some form of platform engineering. They might just need to work out the kinks. Below, we’ll define platform engineering as it relates to DevOps and share some best practices to consider when rolling out (or reiterating) a platform engineering plan within your organization.

    State of DevOps and Platform Engineering

    So, what is the state of platform engineering and DevOps? Contrary to the salacious ‘DevOps is dead,’ headlines, Lynch doesn’t believe DevOps is going anywhere. Instead, platform engineering is helping overburdened developers, having emerged at a time when developers and operations seem to be at a crossroads.

    Developers are now responsible for coding, security, deployment and infrastructure maintenance. “Developers are talented and creative, but everyone has only so much capacity,” said Lynch. She believes platform teams can relieve some of these pressures by creating standard builds and common CI/CD pipelines. “I see it as a partnership between the DevOps teams and platform operations teams.”

    Lynch views DevOps as more of a practice than a specific job title. And although developers should have control over their DevOps toolchain, she believes they were granted too much responsibility. For example, the amount of effort required to search the market, talk to vendors, and try out tools can all add up significantly.

    By creating proven patterns and managing tools, platform engineering teams can help narrow down the options, working behind the scenes to ensure the platform is self-serviceable and continually updated. Of course, this partnership requires regularly collecting feedback to ensure platform teams are enhancing development workflows, not hindering them.

    Best Practices for Platform Engineering Adoption

    So, what are some best practices that enterprises can apply to their internal DevOps platforms? Lynch shared some areas platform engineering teams should focus on to improve their process. By following these tips, a platform can enhance DevOps principles within a company.

    • Be open to feedback from developers. As previously mentioned, platform engineering and development should be partners. Platform engineers should seek to gather as much feedback as possible from developers to improve their experiences and ensure the right DevOps tools are acquired.
    • Don’t control everything. As such, a degree of flexibility should be granted—you don’t want to mandate standardized adoption of all tools. Yet, you do want to decrease rogue tooling and sprawl. The platform team should thus work to find a balance and build trust.
    • Maintain a DevOps culture. When people discuss DevOps, they often talk about the tools and technology, not why DevOps came to be. “DevOps is here because we were trying to find a way to allow teams to work better together and be high-performing,” said Lynch. As such, we shouldn’t forget to include the people element here. Platform engineering should reinforce a healthy culture with positive dialogues and ensure work is “a fantastic place to be.”
    • Treat the platform as a product. When delivering internal-facing tools, your developers are your consumers. Thus, it’s a good practice to treat the platform as a product in its own right. This includes publishing roadmaps and eliciting feedback. “The platform team really needs to win the business of their internal customers, or else they’ll buy or use something else, which could cause shadow IT.”
    • Focus on increasing developer experience. As part of viewing internal developers as customers, platform teams should invest in improving the developer experience associated with using the platform. This means reducing friction and integrating new capabilities into preexisting workflows whenever possible.

    Goalposts For Platform Engineering Success

    For any new initiative to be successful, it must have goalposts and be measured in some capacity. The same is true for platform engineering. So, what are some goalposts that IT leaders should have in mind when introducing platform engineering? According to Lynch, some positive end results include collaboration and efficiency, cost analysis, cybersecurity and tooling centralization.

    • Collaboration and efficiency: First, platform engineering should improve work in the best way for everyone, says Lynch. “The people component is always there and should always be a priority,” said Lynch. To reach these goals, she recommends having the right conversations to co-discover bottlenecks in the value stream management of software development.
    • Cost optimizations: Secondly, platform engineering is an opportunity for garnering insight into usage rates, which could inform cloud cost analysis. Platform teams can implement specific policies, such as spinning resources up or down to reduce expenses.
    • Improved security: Platform engineering teams can be the first to test new products, giving them a unique opportunity to synchronize requirements with security teams. They can also take the reins of continually patching dependencies to reduce vulnerabilities.
    • Increasing consistency. In general, another benefit is centralizing tooling control. Because when tens and thousands of developers are each using their own tool, it becomes harder for teams to work together. While some cutting-edge technology organizations may seek to give developers control over the full cycle to increase agility, most will benefit from centralizing procurement and oversight, said Lynch.

    The Human Side of Platform Engineering

    “Platform engineering, when done right, furthers the principles of DevOps,” said Lynch. And interestingly, it may be the case that teams don’t even realize they are already doing platform engineering. Once they realize they are gravitating toward these ideals, it will be an even greater reinforcer for DevOps and DevOps principles, she said.

    Currently, the tech space is in a tricky spot, with rising layoffs and tightening budgets. At many organizations, fewer people are now expected to perform the same high caliber of work. As a result, we need to place more emphasis on retaining the remaining talent and boosting morale. This means caring about what the developer’s experience is like throughout the day and supporting the human side of DevOps. According to Lynch, platform engineering is a natural choice in these circumstances.

  • State of DevOps Report Finds a Rise in Platform Engineering

    State of DevOps Report Finds a Rise in Platform Engineering

    The speed of software development velocity is increasing, and to enable this, platform engineering is emerging to evolve the DevOps practice. Interest in platform engineering is rising—so much so that Puppet emphasized platform engineering in its 2023 State of DevOps Report.

    A whopping 94% of respondents agreed that platform engineering is helping their organizations better realize the benefits of DevOps. The move toward platform engineering has increased over time, bringing benefits such as increased release velocity, reliability and efficiency.

    Below, we’ll highlight some of the key takeaways from the report. We’ll define what platform engineering is and highlight its benefits. We’ll also consider obstacles on the path to embracing a platform-based development culture.

    Platform Engineering Supplants DevOps

    DevOps has been in use for a decade or more at most institutions, maturing steadily with new automation and infrastructure abstraction. The latest evolution of DevOps is a practice known as platform engineering—51% of organizations have already adopted this practice within the last three years and 93% said it’s a step in the right direction.

    Platform engineering is the discipline of designing and building self-service capabilities to minimize cognitive load for developers and to enable fast-flow software delivery.

    In essence, platform engineering is the process of creating standardized, self-service infrastructure that helps developers get up and running quickly. These self-service platforms typically handle capabilities like deployment (43%), operation of infrastructure (43%), monitoring production applications (42%) and security and compliance (42%).

    These shared internal platforms are maintained by a central platform team that embraces a product mindset to iterate the technology based on user feedback.

    State of Platform Engineering

    Platform engineering is relatively new— 27% of platform teams have formed within the last two to three years. Notably, big tech companies hopped on the bandwagon years ago, and non-tech companies are now following. Regardless, 70% believed their timing was “just right.”

    The goals of a platform engineering team are numerous. According to the Puppet report, 54% said problem-solving is the top goal. This is followed by educating and empowering developer teams (47%), setting best practices (46%) and optimizing iteration (44%). A platform engineering team’s scope of work is primarily focused on automating workflows and building and managing infrastructure.

    The ‘correct’ number of internal platforms is a moving target. The majority of organizations (57%) have between two and four platforms, and 30% have five or more internal self-service platforms. This demonstrates that although platform engineering is somewhat centralized, it’s also flexible to the needs of an organization.

    Benefits and Hurdles

    So, what are some benefits as opposed to traditional DevOps? Well, the most common benefit is improved system reliability, at 60%. This tracks, as a standardized platform could reduce human error and missteps in the development workflow. Following this benefit is improving efficiency (59%), speeding up delivery time (58%) and improving workflow and process standards (57%).

    The most common driver of adoption is increasing the speed of delivery. Slight more than half of respondents (51%) said development speed has somewhat increased since the inception of the platform team, and 42% say it’s increased by “a great deal.” Other top drivers include the need to scale, free up engineering time, control infrastructure spending and avoid duplication across teams. Those with more mature adoption tend to be more satisfied with the result, indicating that results increased over time.

    Of course, no new technological introduction comes without its hurdles, and platform engineering is not immune from certain pain points. For example, 34% said that cycle time is slower than expected. This might be due to nascent industry adoption and a lack of best practices.

    Just under one-third (32%) also noticed resistance to platform team adoption, and 32% reported a lack of communication around changes. Organizations will likely have to increase transparency and reduce siloing to encourage productive platform adoption that satisfies the needs of various stakeholders.

    According to the report, product management skills can help realize these goals. Some of the most fundamental product management skills here include strong communication skills (61%), problem-solving expertise (60%) and the ability to foster collaboration (54%). Many companies will also have to convince senior management to invest in platform engineering—nearly half agreed that senior management doesn’t yet understand its value.

    Future Outlook

    Confidence around platform engineering is increasing, especially for teams with mature DevOps approaches. As a result, the practice will likely gain more traction in the coming year—71% of respondents reported their firm plans to hire people with platform engineering experience over the next 12 months.

    Over the next year, organizations that adopt the practice will want to increase awareness of platform capabilities and set expectations across the organization. This will also include setting governance policies and satisfying the needs of specific developers, departments and the organization as a whole.

    Above, we’ve scratched the surface of the current state of platform engineering, its benefits and its pain points. For plenty of other insights, you can pick up a copy of The State of DevOps Report 2023: Platform Engineering Edition from Puppet by Perforce here.

  • Atlassian Aims to Improve Collaboration With DevOps Teams

    Atlassian Aims to Improve Collaboration With DevOps Teams

    At an Atlassian Presents: WorkLife event today, Atlassian furthered its efforts to integrate the tools technology and business professionals use to manage processes.

    Erika Trautman, general manager and vice president of product for Work Management for All at Atlassian, said that as organizations increasingly rely on digital processes to engage customers, partners and employees, the need to facilitate collaboration across a diverse set of knowledge workers is becoming more pressing.

    As part of that effort, Atlassian is now making generally available a Smart Links capability within the Atlassian platform. Smart Links can be used to integrate both Atlassian applications such as Trello and Jira as well as third-party applications. That capability is now critical because there has been a Cambrian explosion of adoption of software-as-a-service (SaaS) applications that all need to be integrated into a common set of workflows, noted Trautman. Otherwise, each SaaS application environment becomes yet another walled garden that prevents any meaningful collaboration, she added.

    In addition, the company announced that Atlas, a directory that tracks projects and simplifies communications across teams using disparate applications, will become generally available next month. DevOps teams, for example, will be able to synchronize Jira events with projects being tracked via the Atlas platform.

    Atlassian has also streamlined the tools it provides to reduce the number of steps required to administer its applications and onboard new users.

    Finally, the company also has launched Atlassian Together, a single subscription offering through which organizations can license its Trello, Confluence, Atlas and Jira Work Management applications delivered via the cloud. An enterprise organization with more than 5,000 users can now, for example, license those applications for $11 per user.

    Atlassian is making a concerted effort to enable organizations to more easily collaborate at a time when they have become more dependent on software than ever, noted Trautman. In many cases, businesses are now trying to adjust to the rapid rate at which software is being delivered thanks to the adoption of DevOps best practices, she noted.

    As such, there is a greater need for integration between tools such as Jira that development teams use to manage projects and a wide range of tools used by knowledge workers residing in various departments, said Trautman. In effect, Atlassian is trying to enable organizations to reduce the “collaboration tax” they incur as workflows span multiple silos, she added.

    In fact, in many cases, organizations are struggling to keep pace with how agile software development teams have become. The irony is that not too long ago business executives routinely complained about how long it took IT organizations to adjust as business conditions changed. Now, many businesses are rolling out additional digital capabilities via software updates that can be made multiple times a month.

    The rate at which organizations will collaborate across departments will naturally vary, but the days when each department was a fiefdom unto itself are coming to an end. The challenge and opportunity may now be determining how modern organizations should be structured once it becomes easier to collaborate.

  • How to Adopt an SRE Practice (When You’re not Google)

    How to Adopt an SRE Practice (When You’re not Google)

    Site reliability engineering (SRE) isn’t a new term or practice. The practice of applying software engineering skills and principles to operations problems and tasks happened even before site reliability engineer was a defined job title. But organizing a proactive approach to building and maintaining software drives long-term success in improving operational efficiency, data-driven roadmap planning and general uptime and reliability. All these advantages account for the widespread adoption of SRE. 

    In this article, I’ll dive into what it takes to get into site reliability engineering, how to adopt it within your own organization and some of the core principles and best practices you’ll need to keep in mind as you move forward in your SRE maturity journey.

    What is SRE?

    SRE, or site reliability engineering, is the practice of applying software engineering expertise to DevOps and operations problems. Often, this means proactively writing code and developing internal applications or services to combat reliability and performance concerns. SRE has been a practice for many years but has been more recently popularized by Google when they first published Site Reliability Engineering: How Google Runs Production Systems in March 2016.

    SRE is a complicated acronym because it both relates to a job title, site reliability engineer,  as well as a general practice adopted by development and IT teams, site reliability engineering. SRE teams are often organized differently depending on the organization and the services they support. Sometimes site reliability engineers (SREs) are sprinkled among development teams. This means they can vocalize concerns during roadmap planning and work closely with QA teams, collaborating on everything from shipping features to staging to managing production environments. 

    Another approach organizes an SRE team as a standalone group. The SRE team as a whole is focused on responsibilities including monitoring applications and infrastructure, establishing reliability metrics, tracking issues with new releases and handling on-call duties. No matter how they are organized, what remains the same for all these SREs is their initiative—reliability and performance.  

    DevOps Vs. SRE

    The rapid adoption of DevOps practices may have you wondering, “How is SRE different from DevOps?” Commonly, the two terms go hand-in-hand. SRE is a practice that doesn’t necessarily have to be part of DevOps culture and adoption but is often implemented by organizations that are further along in their DevOps journey. The further along on your DevOps maturity model, the more likely you are to have bought into SRE practices.

    The conversation isn’t ‘SRE versus DevOps; the conversation is about establishing a proactive SRE team and operating model to help strengthen your DevOps practice. Whether you choose an SRE framework integrated into your development and IT teams, or you want to make SRE a standalone business unit, you need to understand their general responsibilities.

    SRE Team Responsibilities

    Site reliability engineers are responsible for defining what performant and reliable really mean when you talk about your applications, services and infrastructure. The day-to-day tasks can range anywhere from instrumenting new monitoring solutions to building custom apps for technical support teams. They might ship new code to production to fix bugs or deliver new features, or they might respond to incidents in real-time and work closely with support teams to deliver positive customer experiences.

    At the end of the day, site reliability engineers are the key to understanding exactly how customers experience your product and how to track system performance and reliability through your customers’ eyes. SRE teams need to figure out exactly where service boundaries exist between what development teams ship and what customers experience. From there, they need to find ways to monitor reliability and performance concerns in a way that helps your internal teams proactively identify risks and deliver better software. 

    SRE teams spread knowledge across the product and development teams to consistently define reliability across the entire organization. With everyone on the same page, the engineering teams can make data-driven decisions when it comes to releasing new features or improving current experiences in production. 

    Site Reliability Engineering Principles and Practices

    At a very high level, Google defines the core of SRE principles and practices as an ability to ’embrace risk.’ Site reliability engineers balance the organizational need for constant innovation and delivery of new software with the reliability and performance of production environments. 

    The practice of SRE grows as the adoption of DevOps grows because they both help balance the sometimes opposing needs of the development and operations teams. Site reliability engineers inject processes into the CI/CD and software delivery workflows to improve performance and reliability but they will know when to sacrifice stability for speed. By working closely with DevOps teams to understand critical components of their applications and infrastructure, SREs can also learn the non-critical components.

    Creating transparency across all teams about the health of their applications and systems can help site reliability engineers determine a level of risk they can feel comfortable with. The level of desired service availability and acceptable performance issues that you can reasonably allow will depend on the type of service you support as well. SRE principles and practices embrace experimentation and require a dedication to proactively understanding the health of the services they support.

    An SRE Operating and Maturity Model

    You can perform a lot of the responsibilities asked of a site reliability engineer and still have a software engineer job title. So, how do you know how mature your site reliability engineering practice is? Luckily for you, we’ll lay out a quick way to build an effective SRE operating model and track your maturity against it. An SRE operating model typically includes three elements, which you might achieve in phases:

    1. A team (or at least one person) dedicated to the practice of SRE.
    2. Deep integration and influence across product, development and operations teams.
    3. Autonomy to automate workflows and write code for nearly any part of your application or system.

    Your SRE maturity depends on where your organization falls along these three elements of an SRE operating model. If you’ve taken the steps to make an SRE team or hire your first site reliability engineer, you’re at the beginning of your journey. If you have a team and they are a valued part of roadmap discussions, QA, deployment workflows, incident management processes, then you have a somewhat mature SRE practice. 

    An organization only reaches complete SRE maturity when the SRE business unit has the autonomy to automate workflows, build applications, own monitoring and alerting solutions, or interject themselves into nearly any conversation. Vocalizing performance and reliability concerns upfront and having a proactive discussion about those concerns is always better than simply ignoring them until it’s too late.

    Monitoring, CI/CD and Organizational Automation

    Site reliability engineers can and will automate just about anything and everything. If it can proactively detect, remediate or resolve an issue, it needs to be automated. From continuous integration and delivery practices to production environment monitoring, SREs should have some visibility into all of it. If they can identify ways to proactively uncover performance and reliability issues, then they need to have the authority to implement those changes.

    Today’s DevOps and IT capabilities around automation, monitoring, artificial intelligence and machine learning give SRE teams a huge advantage when identifying issues, responding to them and fixing them. Organizations with mature DevOps and SRE practices can catch problems in staging and they can also build automated incident management workflows and self-correcting systems. By determining critical components in your applications and infrastructure, SREs can narrow down the scope of things that can cause major problems.

    The Practice of Service Levels (SLIs, SLOs and SLAs)

    Service levels come into play to help SRE teams communicate the true health of digital products and services to all stakeholders. This is done by identifying and measuring critical components that are key to delivering positive customer experiences. In particular, they need to know when one or more components expose functionality to external customers. We call these intersection points system boundaries. System boundaries are the place where site reliability engineers need to apply service-level indicators and objectives to their metrics in order to tell the real story of system performance and reliability. 

    • Service-level indicators (SLIs) are the key measurements to determine the availability of a system.
    • Service-level objectives (SLOs) are goals you set for how much availability you expect out of a system.
    • Service-level agreements (SLAs) are the legal contracts that explain what happens if the system doesn’t meet its SLO.

    While SREs aren’t always responsible for managing service levels, it often falls within their purview. By tracking SLIs and tying them to SLOs, you can set goals around the performance of a system. Google’s SRE book defines the four golden signals of service levels as latency, traffic, errors and saturation. So, for example, you could look at an API call and track its number of successful/failed requests (the SLI) against a general percentage of requests that need to be successful for customers to have a good experience (the SLO).

    SRE teams will often set strict SLOs on critical components within their applications and services to better understand how strict of an SLA they can agree to with customers. From here, the team can apply error budgets as a way to understand how quickly they must resolve issues in order to stay compliant with their SLOs. Service levels allow teams to aggregate metrics and create a transparent view of uptime, performance, and reliability across the entire organization. At a glance, business leaders can use service levels to monitor compliance across multiple teams, applications, services, etc. to gain a comprehensive understanding of their system’s health.

    Adopting SRE Best Practices

    Adopting SRE best practices and principles won’t happen overnight. It takes time and effort to proactively monitor your teams and systems for performance and reliability concerns. But, in the end, your DevOps teams and especially your customers will thank you for deciding to take advantage of site reliability engineering.

  • Digital.ai Extends Scope and Reach of DevOps Platform

    Digital.ai Extends Scope and Reach of DevOps Platform

    Digital.ai today unveiled the Ascension release of its namesake DevOps platform that adds pre-built metrics and best practices dashboards using a Test Lens capability to more easily employ analytics to identify software delivery bottlenecks.

    In addition, the company has added autonomous testing tools that automatically generate performance and security tests for mobile applications. These tools can be run by DevOps teams, developers or business users. An enhanced Change Risk Prediction capability enables machine learning algorithms to make more extensive use of historical data.

    Finally, Digital.ai has also added integrations with the open source Argo continuous integration/continuous delivery (CI/CD) platform and made available an Agility Sync tool for building custom integration with legacy application environments and opened an online marketplace for integrations.

    Jeff Moloughney, chief marketing officer for Digital.ai, said the goal is to extend the reach of the company’s DevOps platform. At its core, the platform has a value stream management (VSM) and version control software originally developed by XebiaLabs and CollabNet VersionOne before the two companies merged to create Digital.ai. The combined company subsequently acquired Numerify, a provider of analytics tools infused with artificial intelligence (AI), Experitest, a set of testing tools, and Arxan, a provider of an application security platform. The company claims that its platforms are used by more than 1,500 organizations.

    In general, DevOps is entering a new era as organizations realize how dependent they are on software to drive business processes, noted Moloughney. Historically, the adoption of DevOps practices has been driven from the bottom up within most organizations. Now, business and IT leaders are also looking to drive the adoption of a consistent set of DevOps best practices to gain more visibility into software delivery, he added.

    That does not mean, however, that organizations can dictate what tools individual developers can use, so the DevOps platform employed has to be extensible in terms of the available integrations, said Moloughney.

    At the same time, Molougney added that more DevOps processes will be automated using machine learning algorithms. Digital.ai is embedding these algorithms within its core platform to both uncover software delivery bottlenecks and automate routine tasks such as testing.

    It’s not clear to what degree organizations that have already embraced DevOps might be considering replacing their existing platforms but, regardless of size, most of them are encountering issues. On the plus side, the number of organizations that are adopting DevOps practices more broadly is increasing as more digital business transformation initiatives are launched. As those practices evolve, the expectation is many of those organizations will embrace analytics in the form of VSM to better optimize their efforts.

    In the meantime, the rate of DevOps adoption across enterprises will continue to vary widely. Each development team within an organization tends to define a set of DevOps practices based on their own requirements. The challenge, given all the dependencies that exist between those projects, is determining the actual rate of progress the organization is making as each of those isolated initiatives progresses.

  • Best Practices to Improve your Engineering Sprints

    Best Practices to Improve your Engineering Sprints

    What is the purpose of sprints? We break work into sprints so we can more easily monitor, estimate and predict what we want to ship and when. Over the years, engineering teams have gotten used to moving quickly from one sprint to the next, but I think it’s important to be more thoughtful in how we plan and execute sprints.

    Running effective and successful sprints can define whether or not your organization is able to ship high-quality software fast and often. It also gives teams more flexibility and the ability to quickly change something that is not working or, on the other hand, to iterate on a successful technology. 

    After almost 20 years as an engineer, manager and as a CTO, I have found there are several best practices that improve the sprints that are part of my team’s software development life cycle. 

    Sprint Planning

    When planning a sprint, there are several things to keep in mind.

    Make Sure the Work Aligns with Business Priorities

    There are always a number of elements an engineering team can focus on during any given sprint. It’s fairly easy to lose sight of the business’ end goal. Make sure to gut-check to make sure that the sprint goals align with current business priorities.

    Ensure the Workload is Balanced From the Start

    This seems like an obvious statement, but make sure team members have equivalent workloads so you avoid burnout. It is also important to understand how different engineers on the team work—some people work best by selecting their own tasks with items that are unassigned, some pick up unfinished tasks as they finish others and some prefer to be assigned certain tasks because they have specialized skills. 

    Don’t Underestimate the Time That Interruptions/Additions Take 

    Along the same lines as the above, don’t overlook this for several reasons. Without adequate, data-driven insight into the actual state of each team member’s workload, it is almost impossible to gauge what’s happening in real-time. While business-critical sprints are assumed to take priority, side projects, Slack interruptions and meetings can easily consume an engineer’s day. Make sure to proactively check that each member has enough true deep work time without becoming burned out. 

    Don’t Underestimate the Time it Takes to get Things to a Done State

    Sometimes engineering teams only look at the development stage, but the full cycle required for the software to get to the end user might take longer. Allotting the right amount of time to get the job done across all phases of development will help avoid unexpected surprises. 

    When You Get to Mid-Sprint

    Monitor Unplanned Work

    Initial planning doesn’t necessarily take into account the bugs and fire drills that will likely arise after the sprint starts. Pay close attention to any mid-sprint additions and adjust as necessary. There may be team members who have cleared their sprint queue and are capable of taking on this work, but being aware of team members’ workloads at every stage is critical. 

    Do Daily Stand-Ups and Raise Flags as Early as Possible

    This will help anyone encountering roadblocks so a small issue doesn’t turn into a huge bottleneck. If you look closely at your PR cycle time, you will find a ton of insights about where roadblocks often happen. Using historical data during sprints can also go a long way toward helping identify potential bottlenecks, allowing you to proactively avoid friction by delegating reviews or setting out additional time for approvals. 

    Get Ahead of Burnout

    The tech industry has a turnover rate of 13.2%—pretty high when compared to other sectors. But the reality is that it can be really difficult to identify burnout without hard data. Don’t wait until people start speaking out about being overworked: Burnout prevention should be a full-time focus. Before sprints, make sure people aren’t coming in with high levels of fatigue and make sure that overtime doesn’t routinely increase throughout.

    Use Sprint Progress Estimations to Reprioritize

    Each sprint starts off with a set workload, but in reality, every sprint is dynamic. Take a look at each day’s progress and evaluate if your team is ahead of or behind schedule. If you are on track to finish a majority of the work, you can anticipate a successful sprint end. If not, don’t worry; you can always recommend that your team recalculate priorities. Engineering managers can take this time to load-balance some of the heavier work and shift goals to the next sprint as needed. Staying flexible while maintaining a clear view of what is actually happening during the sprint, will inevitably result in success. 

    At the End of the Sprint

    Do a Thorough Sprint Retrospective 

    As a team leader, you should constantly be thinking about how things are going and how people are feeling. If you can, use hard data to look at things like time allocation, PR throughput and any items added mid-sprint. Taking notes throughout the sprint will help remind you of any issues that came up, and you can discuss ways to improve for next time. Each sprint is a chance to learn; the more you learn, the better the sprint and the better the product will be for end users. 

  • Source Code Management Best Practices

    Source Code Management Best Practices

    Source code is, of course, the foundation of any software development practice. Managing that source code is the first task tackled by modern software development pipelines, with all subsequent stages of the pipeline dependent on the source code for their success and functionality. Thus, it is critical that source code is properly managed without introducing bottlenecks or defects into the delivery pipeline. To do so, there are some best practices to follow when interacting with source code.

    Regularly Commit Changes

    While this may seem obvious, commits often are done sparingly, as most developers tend to commit only after a significant change. The commit frequency depends on the workflow and it is acceptable to commit only after major modifications if that commit triggers another task.

    However, it is better to commit changes—even minor ones—regularly. It not only keeps an auditable history of changes but also enables easy comparisons between commits for troubleshooting and ensures that the code is up to date with the version control system (VCS). However, it does not mean that users should make commits for every small change; instead, adapt a development style that includes frequent commits as part of the workflow. Even if there are many commits, most VSCs provides users with the option to combine multiple commits to a single commit using a rebase for better management.

    Always Ensure That the Repository is Up to Date

    Most of the time, development is a team effort with multiple developers working on a single codebase. Thus, there will be frequent updates to the source code. So it is always advisable to use git pull or fetch to obtain an updated copy of the code before making any changes. This practice not only mitigates merge conflicts but also helps users to avoid errors while pushing the code, such as ‘failed to push some refs to‘ errors which occur when the local copy of the repository is behind the remote.

    Create Detailed Commit Messages

    Commit messages should be meaningful and include the ability for anyone reading the message to grasp the reason behind the commit quickly. It does not mean that users should create lengthy commit messages; instead, they should describe the commit in a concise yet meaningful manner. The commit message should include why the code was committed and not how it can be identified by comparing the files.

    A good commit message can include details such as bug fix ID or requirement ID, which reference the specifications of that code change in project management platforms like Jira and Azure Boards.

    Review Changes Before Committing

    Always make sure to review the code before making commits—this is imperative if you are committing to a shared repository. Typically, code reviews are a part of the delivery pipeline before merging code with the master branch or a specific repository via a review system or a pull request.

    Code reviews put a second set of eyes on code modifications and help detect any issues in the code. It also acts as a quality control mechanism that helps organically improve the overall code quality while helping to increase the awareness of each code change.

    Implement a Proper Workflow

    A proper workflow with standardized practices must be implemented across the development team even while using a source code management (SCM) tool. Some considerations for a standardized workflow:

    • Decide on a branching strategy and branching best practices like naming conventions for specific development types. For example, using a prefix to indicate the type of development feature or bugfix
    • Ensure proper use of scopes to decide who has access to each repository
    • Make sure the review workflow includes which users have the final say in the review process and proper authorization to merge the changes.

    A properly defined workflow describes the complete source code management cycle from creating new branches to making changes and, finally, merging the code to the master branch after review. This process helps organizations simplify the overall code management by pushing users to follow a specific workflow. This can also help easily onboard new developers.

    Implement Proper Security Practices

    This practice goes hand-in-hand with proper workflows, as security will be one of the primary considerations when defining the workflow. It should start from developer authentication and authorization, which controls who has access and what areas of the source code they have access to. Additionally, security practices should provide visibility into all the user activities with auditable trails to pinpoint any unusual activity in the source code.

    Security practices coupled with a good backup and disaster recovery strategy are vital for the continuous operation of the development pipeline.

    These best practices can help developers manage their source code securely and efficiently. A properly implemented source code management system will lead to fewer issues in the preceding stages of the pipeline and ensure smoother operation in the long run.