devopsy.kinsta.cloud

Category: DevOps Practice

  • We Are Living in an Ephemeral World

    We Are Living in an Ephemeral World

    We are lucky to live and work in the times that we do. Oh, we have our challenges, and I talk about them a fair amount here on these pages. But the things we can do in IT today have not been possible in the past, and the change isn’t over. Soon you will be able to relegate pretty much all of the busywork of IT to automated systems, Note that I did not say “AI/ML”–some of that automation will be, but we throw the term around too much right now, and not all automation will be based on AI/ML. In fact, quite a bit of the best automation is simply rule-based systems that we finally have things standardized enough to write up. And I intend to talk specifically about automation that is not AI/ML today.

    We have reached the point in DevOps where we can automate the entire build/release process using containers. The CI tool, running in a container, can spawn a container that performs the build. Then, when it is done, if it passed, it can spawn another container to run tests, do security scans, etc. Finally, the product can be promoted and released … in a container. Human involvement is only required for approvals and then only for organizations that want checkpoints on the way to production (which is definitely most but not all of us).

    If you take the totality of what is going on out there in DevOps, it is finally catching up to DevOps. The biggest issue will be making it efficient and doable for the DevOps team. To date, by focusing on DevOps, teams have been able to avoid the simple fact that complex deployments on infrastructure that spans environments and uses external resources is hard. By keeping the DevOps side simple (or not part of the DevOps process), developers can crank out code faster, facilitated by DevOps tools and Agile methodologies, and operations can continue to wrangle the beasts of networking, storage, etc. to get the app running in production. Once ops has completed the process, then it can be automated based on steps taken.

    But we now have Software-Defined Everything ™ combined with containers. That means that for many–though not anywhere near all–applications, a commit that updated a constant in a single source file can spawn the entire process in iterative containers all the way through release. That is very cool, and we should take a second to appreciate how far we’ve come in a short time.

    Then we need to talk about those worker containers. They need a ton of centralized logging and reporting. Spinning them up, letting them run and then automatically destroying them is fine—as long as the organization has access to the information it needs to review and analyze the results. In-application tooling will cover the released product and even the test environment, but not an ephemeral build process. By not keeping a ton of build tools running all of the time, the enterprise is, by definition, more secure—the blast radius is reduced if build tools are only available when they’re active, and logging of who/what kicked of a build is kept. But don’t forget the reporting. At build time, our tools know exactly which error failed a build; the container needs to submit a ticket before being destroyed or log heavily. I say, “Always create a ticket,” but I’ve worked in enough environments to know that your reality may not make that possible. As long as notifications occur, it doesn’t really matter how they occur.

    So crank it up; start super-automating. Tool up those build environments, then make them into disposable container instances. Others have been doing this for a while, so there is help available for setting it up, but it is just now taking off, so it is a good time to dive in. And keep rocking it. How the apps are built and deployed is only important in the sense that you are driving the company forward. Keep doing it.

  • Platform Engineering: Creating a Paved Path to Reduce Developer Toil

    Platform Engineering: Creating a Paved Path to Reduce Developer Toil

    This article is part one of a series of three on the boom in platform engineering and its role in reducing cognitive load and creating a better developer experience.

    Developer toil and cognitive load in the cloud-native space is real. The complexity posed by microservices, Kubernetes, and “software-defined everything” almost necessitated that ops needed to solve many of these issues to support cloud-native development.

    Developers taking on full dev life cycle ownership and the requirement to shift left have resulted in cross-functional and operational concerns that have added even more complexity to already challenging workloads and unclear workflows.

    To fuel productivity and consistency and enable developers to build and ship apps faster and more securely, one must understand what a ‘paved path’ might look like and where responsibility for different aspects of the development life cycle resides.

    Supporting the Majority of Developers Without Alienating the Rest

    Most developers want and need paved paths, particularly as paradigms shift; the adoption of cloud-native development being one such shift. These paths must help them reach the “safety” and productivity on which they are measured in their daily output and provide focus for completing their tasks. Freedom to make choices about processes and tools is not a priority because it ends up being noise that adds to the cognitive load and, in most cases, does not serve the company’s software shipping goals. This majority needs tighter guardrails and guidelines to do their work and get their code into the world without having to figure out too many operational parameters for themselves.

    The cloud-native tooling landscape is vast, duplicative and overwhelming. Just trying to select and test tools could be a developer’s full-time job. Likewise, many organizations and developers have reported frustration—they are wasting a lot of time on repetitive work and routine tasks that deliver no business value, such as setting up environments and troubleshooting CI pipelines along with slow feedback loops in the development process.

    As organizations increase pressure to deliver more with less and do so more frequently, these responsibilities make for a poor and unproductive developer experience and a far-from-ideal use of organizational resources. Stack Overflow’s 2022 Developer Survey reported that 62% of respondents spend more than 30 minutes a day fixing problems and looking for answers, leading to somewhere between 333-651 hours of lost time per week across a team of 50 developers. This reinforces a need for upfront investment in the paved-path developer platform, which eliminates much of this frustration.

    It’s worth noting there is also a so-called 1% developer–one who wants to experiment and discover new tools on their own. Defining the developer experience within narrow guardrails ends up stifling this minority, which not only hampers their productivity but could stand in the way of innovation for organizations as a whole.

    Building quality software requires accommodating both kinds of developers. For most development teams, the goal is to give them a clear route to building apps faster and shipping safely by prescriptively paving the path. The path also must provide enough freedom for adventure, allowing developers to stray from the path (within reason and where it makes sense).

    Internal Developer Platforms: Paving Just Enough Path With Platform Engineering

    Platform engineering and internal developer platforms (IDPs) are much-touted approaches to taming the toil. It is a part of the solution, and as it gains traction, it is becoming more mature. That is, instead of developers and development teams taking on full responsibility for every phase of the software life cycle and potentially cobbling together something resembling a developer platform, the industry trend is moving toward more comprehensive, commercial out-of-the-box platforms. Whether or not this addresses the actual needs of the development organization, this shift works because developer experience is now a business-critical requirement, and a centralized developer platform facilitates that experience (with flexibility enough to house all types of developer experience and developers working across industries).

    The developer platform concept, supported by architecture, DevOps and SRE teams, contains the ingredients of transformation both for business and for developer experience. As the concept reaches critical mass and the “productifying” of developer platforms matures, one thing seems likely: By shifting to a platform model, development organizations are moving from ops-first to developer-first Kubernetes. The developer experience becomes defined by its optimized tools and workflows (and enables wider adoption of cloud-native development across larger organizations and broader industries).

    Refocusing Developer Efforts: Abstractions, not Distractions

    Cognitive toil will exist for developers regardless of the tools they’re offered. Technologies grow more complex and change frequently, and there are performance metrics that must be met. Taking the focus away from developers’ primary concerns—coding and shipping software and features—contributes to this toil without generating much return. As much as developers should be able to gain enough understanding of the way their code behaves and the consequences it can have downstream when shipping and running it, they should not need to become DevOps experts as a part of their work. This is why DevOps, despite predictions to the contrary, isn’t dead.

    The paved-path developer platform gives developers exactly that: The ability to shift left without drifting right. The platform provides much-needed abstractions and tools that enable developers to refocus on priority work without distracting or overwhelming them with too many choices.

    Becoming dev-first doesn’t kill off ops, either. Dev-first Kubernetes and a centralized developer platform elevate the importance of the developer experience while continuing to highlight the importance of DevOps in supporting and optimizing the developer experience. This is an entire topic in and of itself, which will be explored in part two of this article series.

  • Where Does Observability Stand Today, and Where is it Going Next?

    Where Does Observability Stand Today, and Where is it Going Next?

    As we move into the second half of 2023, it’s only natural to engage in the annual ‘state of the industry’ reviews and take a general inventory of the past year while looking out into the immediate future. Observability has gained traction among DevOps and platform engineering teams over the past year and is poised to continue that velocity. But my observation is that the current observability model is broken.

    Too Much Data

    Long story short: Many organizations (and engineers specifically) agree that they are sending too much data to their monitoring tools, resulting in an inability to garner valuable insights because the systems are massively overloaded. Sending this high volume of data also costs too much and requires an inordinate amount of time for analysis, which many organizations don’t have. Put simply–more data doesn’t mean more clarity; in fact, we see the opposite.

    Massive adoption of cloud-native technologies such as Kubernetes that heighten the challenge of efficiently monitoring environments is also accelerating this problem. Beyond increasing costs and complexity, many teams seem to feel that their current approach doesn’t create the desired level of observability for Kubernetes, full stop.

    Another major issue we see on the observability landscape relates to the overall maturity of monitoring processes and teams. In many instances, even when organizations have effective tools, they struggle to interpret the data produced into targeted conclusions that their teams can translate into effective actions. And this is further amplified by the outstanding engineering talent shortage because good people are harder to find and retain than ever before.

    We also see a lot of pervasive challenges related to organizations’ desire to use popular open source monitoring tools-especially if they are attempting to do so in an integrated manner or to support rapidly growing deployments at scale.

    Overwhelmed by Observability Data?

    In summary, if you’re overwhelmed by observability data, if complexity and costs grow out of control and you’re also struggling to make sense of the available information and improve your cloud applications, we think that means that the current observability model really is broken.

    To that end, gaining a deeper perspective into end user opinions on these topics, among others, is the precise goal of the annual DevOps Pulse Report, currently in its sixth year and now in the survey stage. We predict that this year’s research project will underline the aforementioned challenges and help frame how teams might seek to address them.

    What types of findings do we hope to unearth? Last year, respondents revealed that despite a growing perception of DevOps maturity, it was actually taking them more time to respond to emerging problems. Roughly 64% of respondents reported that MTTR during production incidents was over an hour, compared to only 47% the previous year.

    Last year’s report also highlighted the reality that monitoring complex environments remains difficult with over 52% of respondents citing Kubernetes, microservices and serverless as their primary challenges in gaining improved observability.

    Both of these previous findings also seemed to underline the growing issues of data overload and spiraling complexity. We’ve also seen previous evidence of a lack of maturity around distributed tracing—a critical observability practice aimed directly at increasing visibility into complex architectures—despite the fact that 75% of respondents listed it as a priority in last year’s research.

    Is your team struggling to make sense of its available monitoring data, or do so in a way that aligns with your current processes or budgets? Hopefully, the 2023 DevOps Pulse Report, set to publish early this year, will put some numbers behind these assertions and help us all understand how to move forward.

    If you want to participate, you can do so here.

  • Five Great DevOps Job Opportunities

    Five Great DevOps Job Opportunities

    DevOps.com is now providing a weekly DevOps jobs report through which opportunities for DevOps professionals will be highlighted as part of an effort to better serve our audience.

    Our goal in these challenging economic times is to make it easier for DevOps professionals to advance their careers. Of course, the pool of available DevOps talent is still relatively constrained, so when one DevOps professional takes on a new role it tends to create an opportunity for others.

    The five job postings shared this week are selected based on the company looking to hire, the vertical industry segment and, naturally, the pay scale being offered.

    We’re also committed to providing additional insights into the state of the DevOps job market. In the meantime, for your consideration:

    LinkedIn

    Apple
    San Diego, California
    Site Reliability Engineer (SRE) – Video
    $115,000 to $217,500

    SimplyHired

    ScientiaMobile
    Reston, Virginia
    Senior DevOps Engineer
    $110,000 to $150,000

    Monster.com

    Larsen & Toubro Infotech Ltd.
    Farmington, Connecticut
    DevOps Engineer
    $110,000 to $130,000

    Indeed.com

    TherapyNotes.com
    Remote
    Senior DevOps Engineer
    $105,000 to $155,000

    Dice.com

    MRoads
    Dallas, Texas
    Platform Engineer – SRE
    $100,000 to $120,000

  • Azure Migration Strategy: Tools, Costs and Best Practices

    Azure Migration Strategy: Tools, Costs and Best Practices

    Azure migration refers to the process of moving applications, data and other resources from on-premises or other cloud environments to Microsoft Azure, which is a cloud computing platform and service offered by Microsoft. 

    Azure migration can include migrating virtual machines, databases and storage to Azure, as well as re-architecting applications to take advantage of Azure services. Typically, the goal of Azure cloud migration is to take advantage of the scalability, security and cost benefits of the Azure platform.

    4 Azure Migration Tools

    Here are several tools provided by the Azure cloud which can help you migrate applications to the Microsoft cloud.

    Azure Migrate

    Azure Migrate is a service offered by Microsoft as part of Azure that helps customers discover, assess, and migrate their on-premises workloads to Azure. It provides a centralized hub for the discovery and assessment of on-premises workloads and helps customers plan and execute their migration to Azure. Azure Migrate includes the following features:

    • Discovery: Azure Migrate discovers on-premises servers and their dependencies and creates an inventory of them.
    • Assessment: Azure Migrate assesses the readiness of on-premises servers for migration to Azure and provides recommendations for target Azure resources.
    • Migration: Azure Migrate helps customers plan and execute their migration to Azure using various migration tools such as Azure Site Recovery, Azure Database Migration Service and Azure Data Box.

    By using Azure Migrate, customers can move their workloads to Azure with confidence, knowing that they have a comprehensive plan and the right tools in place to ensure a successful migration.

    Azure Database Migration Service

    Azure Database Migration Service (DMS) is a fully managed service offered by Microsoft as part of Azure that simplifies and accelerates the process of migrating on-premises databases to Azure. The service supports a variety of databases, including SQL Server, Oracle, MySQL, and PostgreSQL.

    Azure DMS can perform both online and offline migrations, depending on the requirements and availability of the source database. Online migrations are performed while the source database is still in use, while offline migrations require the source database to be taken offline during the migration process.

    The service supports several migration scenarios:

    • One-time migration: Move a database to Azure once and leave it running in Azure
    • Lift and shift: Migrate an on-premises database to Azure and keep the same database engine running in Azure
    • Change the engine: Migrate an on-premises database to Azure, but change the database engine running in Azure to a different one.

    The Azure DMS provides a simple and efficient way to move your on-premises databases to Azure with minimal downtime, and it’s integrated with Azure Migrate for a holistic migration experience.

    Azure Data Box

    Azure Data Box is a service offered by Microsoft as part of Azure that allows customers to transfer large amounts of data to Azure for storage and processing. The service is designed for customers who have a large amount of data that needs to be transferred to Azure, but have limited network bandwidth or other constraints that make it difficult to transfer the data over the Internet.

    Azure Data Box provides customers with a physical device (called an “Azure Data Box”) that they can use to transfer data to Azure. The device is shipped to the customer, who can then copy data to the device and ship it back to Microsoft. Microsoft will then transfer the data to the customer’s Azure storage account.

    There are several different types of Azure Data Box, each with different storage capacities and data transfer speeds:

    • Azure Data Box: 100TB storage capacity, data transfer speed of 100Mbps
    • Azure Data Box Disk: 40TB storage capacity, data transfer speed of 40Mbps
    • Azure Data Box Heavy: 1PB storage capacity, data transfer speed of 100Mbps
    • Azure Data Box Edge: 100TB storage capacity, data transfer speed of 100Mbps and AI and IoT edge computing capabilities.

    Azure Data Box is particularly useful for customers who have large amounts of data to transfer, and don’t have the network bandwidth or other resources to transfer the data over the Internet.

    Azure Synapse Pathway

    Azure Synapse Pathway is a new feature of Azure Synapse, which is a fully managed cloud-based data integration and analytics platform. Azure Synapse Pathway is designed to make it easy for customers to move their existing data integration and analytics workloads to Azure Synapse without requiring significant re-architecting or rewriting of their existing code.

    Azure Synapse Pathway supports several migration scenarios, such as:

    • Lifting and shifting on-premises SQL Server Integration Services (SSIS) packages to Azure Data Factory.
    • Migrating on-premises SQL Server Analysis Services tabular models to Azure Analysis Services.
    • Migrating on-premises Power BI reports to Power BI in Azure Synapse.

    Azure Synapse Pathway also provides a migration tool that helps customers to move their existing data integration and analytics workloads to Azure Synapse, with minimal changes to their existing code. This allows customers to take advantage of the benefits of Azure Synapse without significant re-architecting or rewriting of their existing code.

    Azure Migration Strategy and Best Practices

    Prepare an Inventory of Digital Assets

    Understanding your digital estate is an important part of creating an effective Azure migration strategy. A digital estate refers to the collection of digital assets and resources that an organization uses to run its business, including applications, data, and infrastructure. In order to migrate your digital estate to Azure, you need to understand what you have and where it’s running. This information can be used to create a migration plan that is tailored to your specific needs.

    To understand your digital estate, you should start by creating an inventory of your applications, data, and infrastructure. This inventory should include information such as:

    • The name and purpose of each application
    • The technology stack used by each application
    • The location of the application and its data
    • The dependencies between applications and data
    • The number of users and usage patterns for each application
    • The storage and compute requirements for each application

    Once you have an inventory of your digital estate, you can use it to create a migration plan that takes into account the specific needs of each application, as well as any dependencies or constraints that need to be considered.

    Best practices for inventorying your digital assets:

    • Start by identifying the most critical applications and data that need to be migrated first.
    • Assess the readiness of your existing infrastructure and applications for migration to Azure.
    • Understand the dependencies between your applications and data, and plan for them accordingly.
    • Identify any constraints or limitations that need to be considered when planning your migration.

    By understanding your digital estate and creating a migration plan that takes into account the specific needs of your applications and data, you can ensure that your migration to Azure is successful and that your business is able to continue to operate during and after the migration.

    Estimate Monthly Workload Costs

    Estimating the monthly costs of your workloads is an important part of creating an Azure migration strategy. By understanding the costs associated with running your workloads in Azure, you can make informed decisions about how to migrate your workloads and how to optimize your cloud costs after migration.

    To estimate the monthly costs of your workloads in Azure, you can use the Azure pricing calculator. The pricing calculator allows you to estimate the costs of running various Azure services, such as virtual machines, storage, and networking, based on the specifications of your workloads. You can also use the Azure TCO calculator to estimate the total cost of ownership of your workloads in Azure, taking into account costs such as data transfer, management, and support. There are multiple other Azure calculators you can use to estimate storage costs, networking costs, and more.

    Image Source: Azure

    When estimating the costs of your workloads in Azure, it’s important to consider the following factors:

    • The size and number of virtual machines required to run your workloads
    • The storage and network requirements of your workloads
    • The number of users and usage patterns of your workloads
    • The services and features required to run your workloads, such as Azure Backup or Azure Security Center

    It’s also important to consider how the costs of your workloads may change over time, such as how the number of users and usage patterns may change or how the costs of Azure services may change.

    Best practices for estimating the monthly costs of your workloads in Azure:

    • Use the Azure pricing calculator and TCO calculator to estimate the costs of running your workloads in Azure.
    • Understand how the costs of your workloads may change over time and plan accordingly.
    • Optimize your costs after migration by using Azure cost management and Azure reservations.
    • Continuously monitor your costs and usage to ensure you’re getting the most value from your investment in Azure.

    By understanding the costs associated with running your workloads in Azure, you can create a migration strategy that is optimized for cost and that enables you to get the most value from your investment in Azure.

    Migrate Workloads and Optimize

    Migrating your workloads to Azure and optimizing them for the cloud environment is an important step in creating an Azure migration strategy. Once you have a clear understanding of your digital estate and have estimated the costs of running your workloads in Azure, you can begin migrating your workloads and optimizing them for the cloud.

    There are several ways to migrate workloads to Azure, depending on the type of workload and the migration requirements. Some common migration methods include:

    • Lift and shift: This method involves moving an existing workload to Azure without making any changes to the workload. This is typically used for simple workloads that do not require significant changes to run in the cloud.
    • Re-architecting: This method involves making changes to the workload to take advantage of the features and capabilities of Azure. This may include re-architecting the application to suit the target Azure service, such as Azure SQL Database or Azure Storage.
    • Refactoring: This method involves making changes to the workload to optimize it for the cloud environment. This may include refactoring the application using cloud-native patterns and practices, such as microservices or containerization.

    Once your workloads are migrated to Azure, you can optimize them for the cloud environment by:

    • Scaling your workloads to match demand.
    • Using Azure services and features to improve the performance and security of your workloads.
    • Monitoring and managing your workloads to ensure that they are running efficiently.
    • Optimizing costs by using Azure cost management and Azure reservations.

    Best practices for migrating your workloads and optimizing for the cloud environment:

    • Start by migrating your most critical workloads first.
    • Understand the migration method that is best suited for your workloads and plan accordingly.
    • Use Azure services and features to improve the performance and security of your workloads.
    • Continuously monitor and manage your workloads to ensure that they are running efficiently.
    • Optimize your costs by using Azure cost management and Azure reservations.

    By following these best practices, you can ensure that your workloads are migrated to Azure and optimized for the cloud environment, which will help you to get the most value from your investment in Azure.

    Monitoring

    Monitoring is an important part of creating an Azure migration strategy and ensuring that your workloads are running efficiently in the cloud. By monitoring your workloads, you can identify and address issues that may arise during or after migration and ensure that your workloads are meeting the needs of your business.

    Azure provides a variety of tools and services for monitoring your workloads, including:

    • Azure Monitor: A service that provides a unified monitoring solution for Azure resources, including virtual machines, storage, and networking.
    • Azure Log Analytics: A service that allows you to collect and analyze log data from your workloads.
    • Azure Application Insights: A service that allows you to monitor the performance and availability of your applications.
    • Azure Security Center: A service that provides security monitoring and management for your workloads.

    When monitoring your workloads in Azure, it’s important to consider the following factors:

    • The performance and availability of your workloads
    • The usage and costs of your workloads
    • The security and compliance of your workloads

    It’s also important to consider how your monitoring needs may change over time, as your workloads and usage patterns may change.

    Best practices for monitoring your workloads in Azure:

    • Use Azure Monitor, Log Analytics, Application Insights, and Security Center to monitor your workloads.
    • Continuously monitor your workloads to ensure that they are running efficiently.
    • Use Azure alerts to notify you of issues that may arise.
    • Use Azure dashboards to create custom views of your monitoring data.
    • Continuously assess and monitor your digital estate to ensure that it’s up-to-date and that your migration plan is still valid.

    By monitoring your workloads in Azure, you can identify and address issues that may arise during or after migration, and ensure that your workloads are meeting the needs of your business. This will help you to get the most value from your investment in Azure and ensure a successful migration.

    Conclusion

    In conclusion, migrating to Azure is a great way to take advantage of the scalability, security, and cost benefits of the cloud. However, migrating your workloads to Azure requires a well-planned strategy that takes into account the specific needs of your business and your digital estate. 

    To create an effective Azure migration strategy, it is important to understand your digital estate, estimate the costs of running your workloads in Azure, choose the right strategy, and optimize them for the cloud environment. 

    Additionally, monitoring your workloads in Azure is crucial to ensure that they are running efficiently, and to address any issues that may arise during or after migration. By following best practices and using the tools and services provided by Azure, you can ensure a successful migration and get the most value from your investment in Azure.

  • Blameless Integrates Incident Management Platform With Opsgenie

    Blameless Integrates Incident Management Platform With Opsgenie

    Blameless this week announced it has integrated its namesake incident management solution with the Opsgenie alerting platform from Atlassian.

    Aaron Lober, director of product marketing for Blameless, said the Blameless platform is now integrated with both Opsgenie and PagerDuty, the two most widely used platforms for managing alerts.

    Blameless provides an incident management platform that normalizes data collected from a wide range of DevOps tools and platforms to determine how best to respond to different incident classes. Armed with those insights, it then becomes possible to quickly assemble the best team to respond to an event, said Lober.

    That capability reduces stress because Blameless also surfaces recommendations for resolving incidents based on the data it has collected, he added.

    The Blameless platform is tailored to organizations that have hired site reliability engineers (SREs) to manage IT operations using DevOps best practices, noted Lober. The overarching goal is to eliminate rote processes that, over time, conspire to burn out incident management teams that often experience high turnover rates, he added.

    Opsgenie has been connected to the Blameless platform via the service catalog and alerting functions. Blameless users responding to or managing an incident can escalate issues to any teams defined within Opsgenie. The service catalog and alerting data generated throughout the life cycle of an incident is automatically captured and stored on the Blameless Incident Timeline. That capability makes it possible to identify appropriate service owners and the escalation protocol that was followed by that team. Blameless can also display the actual user name of each responder that has been notified.

    When accessing Blameless via Slack or Microsoft Teams, Opsgenie can automate both when an incident channel is created and pre-select relevant services that should be made available. Alerts can also be manually triggered using commands from the Blameless chatbot for those who prefer to trigger alerts from within Slack or Teams themselves.

    As IT environments become increasingly more complex, more organizations are making a concerted effort to apply DevOps best  practices to incident management. The goal is to reduce downtime and increase overall resiliency at a time when organizations are more dependent on software than ever. As part of that effort, IT teams should identify the repetitive processes that span multiple incidents and then move to automate as many as possible.

    In the longer term, it’s not clear what impact artificial intelligence (AI) will have on incident management. At some point, though, IT teams should expect their efforts to be augmented by machine learning algorithms capable of identifying issues before they balloon into a major crisis. However, there will always be some human intervention required to manage incidents for the foreseeable future, noted Lober.

    In the meantime, most IT teams should continue to expect the unexpected at a time when IT environments have never been more dynamic and change multiple times a day.

  • OpenAI Hires 1,000 Low Wage Coders to Retrain Copilot ¦ Netflix Blocks Password Sharing

    OpenAI Hires 1,000 Low Wage Coders to Retrain Copilot ¦ Netflix Blocks Password Sharing

    Welcome to The Long View—where we peruse the news of the week and strip it to the essentials. Let’s work out what really matters.

    This week: ChatGPT darling OpenAI wants people to write code in English, and the unintended consequences of blocking shared accounts. (more…)

  • Red Hat Brings Ansible Automation to Google Cloud

    Red Hat Brings Ansible Automation to Google Cloud

    Red Hat today announced that the Red Hat Ansible Automation Platform is now available on Google Cloud to simplify the management of yet another cloud platform.

    Using the Ansible Automation Platform, it’s now possible for IT teams that invoke Google Cloud to access pre-integrated services such as Google Virtual Private Cloud, security groups, load balancers and compute and instance groups.

    Pete Cruz, head of product and technical marketing for Red Hat Advanced Cluster Management for Red Hat, said Google Cloud—alongside existing support for rival cloud services—will make it simpler to consistently apply automation across multiple cloud and on-premises IT environments.

    In addition, IT teams can also count the consumption of the Ansible Automation Platform toward any existing enterprise spend agreements with Google.

    Ansible has emerged as a de facto standard for IT automation and is widely embraced by IT vendors and cloud service providers. That level of support has fostered the creation of a level of automation content that is now unrivaled by any other automation framework, noted Cruz.

    In the meantime, the pressure to automate IT processes is building during uncertain economic times. As IT environments continue to become more complex—thanks, in part, to the rise of cloud-native applications—many IT organizations are finding it more challenging to convince business executives to hire additional full-time employees. As a result, many of those IT organizations are now looking to automate as many processes as possible as more workloads are distributed from the network edge to the cloud.

    Rather than create a single playbook to automate every IT process on an end-to-end basis, organizations generally mix-and-match content created by others to automate workflows across highly distributed computing environments, noted Cruz.

    The biggest challenge when automating IT processes is simply overcoming inertia. Many processes are still manual simply because they always have been. In other cases, IT teams are still relying on undocumented scripts created by IT administrators who have left the organization. As the IT environment continues to evolve, it’s not likely those scripts will continue to scale.

    One way or another, IT is becoming more automated. The irony is that after using IT to automate almost every business process imaginable, it’s only recently that IT teams are getting around to automating their own workflows. Many of the rote tasks that used to be performed by entry-level personnel have been steadily disappearing thanks to automation.

    DevOps teams, of course, have always been ruthlessly committed to automating as many IT processes as possible in the name of efficiency. The rise of Ansible, however, makes it easier for every IT organization to achieve that same goal because they don’t have to wait for a specialist to create a playbook to automate a process. An Ansible playbook that can be easily customized or extended often already exists. The real issue may simply be knowing where to look for it.

  • Three Trends That Will Transform DevOps in 2023

    Three Trends That Will Transform DevOps in 2023

    The onset of COVID-19 accelerated digital transformation-related initiatives like cloud migration, microservices, observability and CI/CD, giving long-delayed projects the budget and staffing they needed to build momentum. After almost three years of increased investment and accelerating progress, CTOs and software development teams are navigating how to enable the improvements in time-to-market and customer experience that they set out to achieve. In 2023, we’ll see enterprise DevOps mature and expand its business impact by placing a central focus on quality, delivering improved time-to-market, customer satisfaction and productivity.

    DevOps Matures as a DX Enabler

    The past three years have seen massive growth in cloud computing, observability and pipeline automation as enterprises raced to deliver new, high-quality digital experiences to consumers. Cloud spending alone increased from $270 billion in 2020 to 397.5 billion in 2022, according to Gartner. With new technologies in place, the focus shifts to processes that deliver business agility and efficiency.

    Under increased pressure to deliver faster, businesses need to revisit, evolve—and, in some cases, undo—decades of risk and change management practices built around brick-and-mortar experiences and long software release cycles.

    As digital engagement increases, so will expectations for software quality and scalability. We’ll need to invest in new tools and practices—from testing to observability and beyond—to ensure that we’re meeting or exceeding those expectations.

    The economic uncertainty will compel us to look for ways to improve productivity and decrease waste. Quality practices are central to both. We’ll continue to look for ways to shift left and catch defects early in our pipeline when they are the least expensive to fix. We’ll want to optimize our build, integration, testing and deployment processes to identify bottlenecks and causes of rework that drain the productivity of so many teams daily.

    DevOps, with its emphasis on automation and collaboration, will connect the dots between operational goals, technological investments and improved software. Established frameworks like DORA give software development organizations concrete ways to measure the efficiency and effectiveness of delivery pipelines. The key metrics outlined by DORA—deployment frequency, lead time for changes, change failure rate and time to restore service—enable teams to align on where they can improve development practices and quality practices.

    DevOps Will be Measured by Business Success

    The value of digital transformation and DevOps will also extend even further across the enterprise as CTOs evaluate DevOps practices with structured assessments such as DORA. With several years of accelerated digital transformation under our belts, the software development industry can build out a broader definition of DevOps performance over time. The DORA framework sets the stage for understanding how engineering practices impact the customer experience and, in turn, company growth. In one example, Liberty Mutual, a company prides itself on exemplary customer service, is already seeing how DevOps performance connects to business success.

    Architect Justin Robinson notes in their roundtable on data-driven decision-making and DORA metrics: “For organizations, this is improved organizational performance. If we want to keep investing in great tech and serving our customers as best as possible, [we need] behaviors and practices that we can adopt to continuously get better and hit on every front here. Happier engineers, a better culture and a better bottom line for our company.”

    In 2023, more enterprises will harness DORA to measure DevOps performance and its impact on company growth.

    Customers Will Reap DevOps Benefits

    We’re already seeing how metrics like deployment frequency can support an organization’s ability to attract and retain users. According to the 2022 Testing in DevOps Report, teams that increased deployments by 50%-100% were two-and-a-half times more likely to report high user satisfaction. The ability to innovate—without introducing defects to users—is the next frontier in DevOps success. Quality engineering will serve as a safeguard for DevOps teams so they can deliver new features faster and with confidence.

    The Testing in DevOps Report also illustrated the growing connection between digital transformation, DevOps and quality engineering. A full 88% of teams with highly automated pipelines and mature DevOps practices reported high test coverage, which in turn is connected to high customer satisfaction. Test coverage, a core metric for quality assurance teams, is becoming the connection between code and customers throughout development. Quality teams will expand their testing strategies to cover more aspects of the customer experience, including non-functional attributes like performance and accessibility. Automation, particularly automated testing embedded in CI/CD pipelines, will allow teams to adopt these additional quality factors without slowing deployments.

    Connecting Engineering Transformation to Business Success

    2023 is shaping up to be an unpredictable year in technology. But with the momentum propelling digital transformation to an enterprise-level priority, engineering leaders have the data and the tools to shift focus from DevOps processes to transformative outcomes. DevOps maturity, measured and improved with DORA, will help organizations extract more value from their digital transformations by supporting greater innovation and better customer experiences. Quality engineering, augmented with new testing strategies that measure more of the customer experience, will play a critical role in this evolution by enabling development teams to move faster with confidence.

  • Automation Challenges Holding DevOps Back

    Automation Challenges Holding DevOps Back

    A survey of 500 IT and engineering leaders in the U.S. published today finds organizations build an average of 6.7 internal applications per year themselves in addition to investing in 7.7 separate platforms to build applications.

    Commissioned by Internal, a provider of platforms for building application development environments, the survey also found organizations are spending 40% of their time building and maintaining these internal tools and workflows. While more than three-quarters (76%) of respondents said that this work is necessary, it distracts from building applications that drive business revenue. A total of 82% said a lack of true flexibility and customization is the top reason so much time is spent on building and managing internal tools and workflows.

    A full 94% of respondents said they want a more collaborative platform that enables technical and non-technical users to build internal tools/workflows faster together. The survey also found 91% of respondents believe a platform that encompasses application development and workflow automation would save development time.

    Internal CEO Arisa Amano said one thing that holds IT teams back is that managing the platforms used to build applications requires too much programming expertise. Internal is making a case for a graphical approach to building application development environments so IT administrators can easily construct them using a drag-and-drop interface on behalf of an application development team. That approach makes it possible to allocate more programming resources to building applications, she said.

    More organizations than ever are revisiting how applications are built as the number of projects increases despite an uncertain global economy, noted Amano. Organizations of all sizes are reluctant to hire additional full-time engineers even as digital transformation initiatives demand a growing number of custom applications, she added. The only way to effectively address those conflicting goals is to adopt a platform that automates much of the workflow associated with building an application development environment, said Amano.

    It’s not clear how willing DevOps teams are to replace the platforms they use today to build those environments, but many organizations are still relying on bespoke platforms they built and continue to maintain. Many of those platforms are made up of a collection of tools that DevOps teams cobbled together over the years. However, many of the software engineers that initially built those integrations may have since moved on and the documentation left behind is likely to be scant, noted Amano.

    Most DevOps teams in 2023 will continue investing in higher automation levels to accelerate application development. The quandary they all face is whether to continue to invest in the platforms they have already or migrate to a platform that makes it simpler to build and maintain an application environment using automated workflows built and maintained by a vendor. Regardless of what they choose, it’s certain the days when organizations could afford to throw engineers at every integration challenge have come to an end.