devopsy.kinsta.cloud

Tag: Azure DevOps

  • Why Every DevOps Team Needs a Spot Instance Strategy

    Why Every DevOps Team Needs a Spot Instance Strategy

    Most DevOps teams use the public cloud extensively and focus a lot of energy on reducing cloud costs. According to one estimate, U.S. businesses spent $14.1 billion in 2019 just on wasted, unused cloud resources. Spot instances are one of the most important ways to reduce the cost of public cloud services.

    Spot instances are not a new development. Amazon was the first to announce this pricing option in 2009, turning cloud computing into a market influenced by the dynamics of supply and demand. Microsoft Azure took more than a decade to follow suit, announcing its spot virtual machines offering in May 2020.

    The spot pricing model is simple on the surface but can be complex to implement. Spot instances are how cloud providers sell their unused capacity. When compute instances are not ordered by anyone via the regular, on-demand pricing model, the cloud provider puts them up for auction at very favorable prices, usually around 10% to 20% less than the on-demand cost.

    However, it’s not so simple to get this 80% to 90% discount. When another cloud customer requests the spot instance, the cloud provider sends a notification, and you need to immediately move your workloads before the instance is terminated. This makes it difficult to use spot instances for stateful, database-driven applications, or those that require high availability.

    This is not the only challenge. The market price fluctuates and is much more volatile than on-demand pricing. This means the level of discount that can be achieved over time is relatively unpredictable.

    However, there are at least three ways savvy DevOps teams can make great use of spot instances:

    1. Automation—DevOps teams are proficient with a range of automation tools, including configuration management, infrastructure as code (IaC) and cloud provider auto-scaling tools. These can be used to manage workloads in clusters and automatically fail over when a spot instance terminates.
    2. Dev/test environments—DevOps teams are responsible for setting up development and testing environments, which are extremely suitable for spot instances because they can usually tolerate brief interruptions, and, in many cases, are not stateful.
    3. CI/CD jobs—running jobs on Jenkins, GitLab and similar tools can be easy to scale on spot instances. These jobs are stateless, and if an instance drops, it’s easy to rerun the job on another.

    AWS Spot Instances

    AWS Spot Instances let you buy unused Amazon EC2 computing power at a significantly discounted price. You can specify a price, and when a spot instance is offered at that price, it is launched with the Amazon Machine Image (AMI) of your choice.

    How Spot Instances Work on AWS

    Spot instances are priced at a variable spot price, which is adjusted according to supply and demand conditions. To see current rates, use the AWS Spot Instance Advisor.

    You create a spot instance request, specifying what instance types you are interested in, and the availability zones (AZ) in which they should run. If capacity is available, and their current price is less than your maximum bid, instances are launched.

    Spot instances continue running until:

    • Capacity is no longer available (because the instances were requested by on-demand customers).
    • The price has risen over your maximum bid.
    • You request to terminate the instance, or it is automatically terminated by auto-scaling.

    You can also order spot instances with a predefined duration—you then pay a static hourly rate for that entire duration (even if market price changes in the interim).

    Automation Options

    You can automatically scale instances on Amazon EC2, including both on-demand instances and spot instances in a single auto-scaling group. If spot instances are not available when scaling up, the group can use regular on-demand instances.

    Amazon also supports mixing in reserved instances and savings plans (additional ways to save on on-demand instances by committing to a certain period of time or total capacity). So you can combine multiple saving methods in the same auto-scaling group.

    You can improve availability by deploying applications across multiple instance types running in multiple AZs. By allowing multiple instance types, you tap into multiple pools and increase the chances of obtaining a spot instance when you need it.

    Azure Spot Instances (Spot VMs)

    Azure offers Spot VMs that give you access to unused compute capacity. You can request a single spot VM, or launch multiple spot VMs using an Azure VM Scale Set (VMSS). Spot VMs replaced the previous Low Priority VMs feature, which let you purchase VMs that were in low demand on Azure for a reduced price.

    The spot price of VMs on Azure depends on the total capacity available for that specific instance size and SKU (instance type) in the Azure region. Azure commits to changing pricing slowly—avoiding sudden spikes—to maintain pricing stability and make it easier to manage budgets.

    Like on Amazon, discounts fluctuate significantly, and spot VMs can be up to 90% cheaper than the base price of the same VM.

    How Spot VMs Work on Azure

    The Azure Portal provides access to Azure spot VMs. When you create a spot VM, you can see the current price for the selected region, image and VM size. For consistency, prices are always in U.S. dollars, even if you use a different base currency for billing.

    There are two options for eviction of spot VMs—you can choose the condition on which spot VMs will be evicted:

    • Maximum price eviction—You set a maximum bidding price, and when the spot VM rises over that price, it is evicted.
    • Capacity eviction—You always pay the current price of the VM (without setting a maximum price), and when Azure does not have sufficient capacity of the requested VM type, your VM is evicted.

    When a VM is evicted, Azure applies an eviction policy called Stop / Deallocate. This means the instance is paused, but attached disks remain, and you are still charged for them. When the price goes down or capacity becomes available, the instance is restarted and continues working on the same disk data.

    Automation Options

    Azure provides virtual machine scale sets (VMSS), which can automatically increase or decrease the number of VMs running your application. You can create a scale set that includes spot VMs, and as your application scales, more spot VMs will be added as they become available. Spot scale sets operate in a single fault domain and do not guarantee high availability. Unlike AWS, Azure currently does not allow you to mix on-demand VMs and spot VMs.

    Both Amazon and Azure provide robust capabilities for cost savings using spot instances. Azure’s offering is newer and provides less-advanced bidding and auto-scaling capabilities, but these are expected to be added as the service matures.

    Whether DevOps teams choose to run in AWS, Azure, or both, they literally cannot afford to ignore spot instances, especially for low-criticality workloads like dev/test and CI/CD job execution.

  • DevOps as a Service: Migrating Your Entire DevOps Stack to the Cloud

    DevOps as a Service: Migrating Your Entire DevOps Stack to the Cloud

    DevOps as a Service (DaaS) enables teams to centralize their tooling and workflow into one unified solution. This solution typically contains all necessary DevOps pipeline features, which replace legacy tooling. This article explains how DaaS works, including pros and cons, and an example showing how to migrate from Azure DevOps Server to Azure DevOps in the cloud.

    What Is DaaS?

    DaaS is a managed, cloud-based service that provides a unified DevOps toolchain. All or most of the tools necessary for a continuous integration/continuous delivery (CI/CD) pipeline are collected in a single platform. This platform is then provided to teams with the support of dedicated DevOps professionals.

    The goal of DaaS is to enable organizations to focus on developing and delivering software without having to worry about managing or maintaining tools. It is designed to abstract away the intricacies of tool integration, deployment and maintenance. This enables teams to focus on higher-level tasks, and outsources significant manual effort.

    DaaS can provide small to medium businesses with enterprise-level infrastructure and tooling without needing to hire more staff or build extensive expertise. Instead, these businesses can rely on the experience of DaaS providers to create, deploy and optimize a pipeline for them. These pipelines are created to be intuitive and to facilitate the smooth collaboration of all team members. 

    DaaS Pros and Cons

    If you are looking to migrate your development processes to the cloud, there are several migration strategies you can use. One involves moving existing tools to cloud resources as microservices, which you must then integrate. The other may involve scraping existing tools and adopting a DaaS. Whichever the case, you need to understand the pros and cons of DaaS before you can make the right choice for your organization.

    Pros of DaaS:

    • Managed services provide access to dedicated DevOps engineers for support, expertise and optimization.
    • Provider teams have experience across a range of organizations and industries, and can customize solutions for you based on that experience.
    • DaaS teams can provide mentorship and guidance which helps your team improve their own skills. This enables teams to learn from the mistakes of others and prevents repeating mistakes.

    Cons of DaaS:

    • The importance of pipeline management and availability requires you to partner with a service provider that is reliable and can ensure zero to minimal interruptions.
    • Outsourcing infrastructure demands and providing access to source code and other sensitive data and systems. This can create a security risk that you are not in control of.
    • DevOps teams still need to understand the tools being implemented and the basics of how to manage and use those tools. 
    • Managed services often cost more than self-implemented pipelines and the costs may not be offset by an in-house reduction in IT responsibilities.

    DevOps Cloud Migration Example: Migrating from Azure DevOps Server to Azure DevOps in the Cloud

    Azure DevOps Server, formerly called Team Foundation Server (TFS), is a server hosted on-premises that enables you to build DevOps pipelines. While based on similar services to those offered in the Azure cloud, it requires you to manage and maintain your infrastructure and provides limited support for your configurations.

    By migrating to the cloud, you can outsource many of your infrastructure responsibilities, focusing instead on producing quality software. Azure DevOps Services are an example of partially managed DaaS that you can use.

    Below is a brief guide for how to transfer your data and projects from DevOps Server. This guide is adapted from the full DaaS guide provided by Microsoft.

    1. Preparation: Planning your migration requires taking an inventory of how you are currently using your tools, what data you need to migrate and how migration will impact your productivity. You also need to determine if you need additional licenses. For this step and those to follow, there is a tool you can use to assist you in the process.
    2. Configure Your Prerequisites: Before you can migrate your pipeline data you need to set-up a few prerequisites. You should synchronize your local Active Directory (AD) with Azure AD for authentication. You also need to set up security and compliance measures to ensure that your infrastructure remains secure.
    3. Update and Validate Your Server: Check to make sure that your server is up-to-date with the latest version of Azure DevOps Server. If you are still using TFS, this may mean a series of updates. Once you’re fully updated, you can use the migrator tool to validate the collection databases of each of your projects. If any errors are found, you need to fix those issues during this phase.
    4. Ready Your Cloud Resources: After all errors are eliminated, you can begin preparing your cloud resources for import. This requires assigning, activating and mapping your DevOps Services subscriptions. You also need to generate a template for your import settings through the migrator tool. These settings should be saved in your Import Specification file. Finally, you need to create a container instance in the data center you want to use your DevOps Services in.
    5. Migrate Your Pipelines: After all of your preparatory and planning stages are done, you are ready to migrate your workloads. During this phase, you should first run a test import to ensure that all configurations are correct and that the import process goes smoothly with no data corruption. This helps ensure that you don’t create unnecessary downtime.

    Provided your test run goes smoothly, you can then proceed to import your data. Once uploaded, you need to assign your organization name, set up billing and redirect your on-premises build servers to the new cloud resources. After you are done, you can resume operations and begin operating your pipelines directly from the cloud.

    Conclusion

    DaaS is typically not just a centralized platform, but also a managed service that comes with certain levels of support and optimization. If you are short on DevOps talent and tooling, DaaS can fill this gap or even help you and your team learn DevOps skills.

    However, since you are giving the DaaS of your choice control over your pipeline, you need to investigate and research. You will be giving your DaaS access to source code and security. Before signing a contract, make sure this service is trustworthy and meets your standards.

  • Pre-Commit Job Workflow in Azure DevOps

    Pre-Commit Job Workflow in Azure DevOps

    Many organizations use a combination of open so Many organizations use a combination of open source and commercial tools and Azure DevOps (formerly called VSTS) and an application lifecycle management (ALM) tool that includes features such as Azure pipelines. This article talks about how to achieve an end-to-end DevOps setup in Azure

    Pre-requisites:

    1. Subscription
    2. Project created with version control selected
    3. Process template selected (Agile/Scrum)
    4. Code repo–one on VSTS and one on GitHub/cloud GitLab
    5. Build server–Use credits from MSDN subscription
    6. Deploy–Azure subscription from MSDN
    7. Other tools–tools supported by VSTS/Azure DevOps
    • The objective of a pre-commit job is to reduce the risk of complexities, duplications and potential bugs in the code to get deployed directly into production.
    • In usual workflows, the build will be generated on regular intervals and released for testing. However, there is a chance of several code quality issues and building deployment issues.
    • With the help of pre-commit job, we can configure triggers to generate builds for every pull request made by developers. Also, this enables the development team to create pull request releases that deploy your pull request code or pull request builds, which help to detect deployment/release issues before the code changes are merged.
    • Pre-commit deployments also help catch deployment/release issues early in the cycle, maintain better code quality and release with higher confidence.

    Configuring Agile Board in Azure DevOps for Project Management, and Tracking Steps to Do It

    Steps:

    With Azure Boards we can plan, track and discuss work across different teams. With Azure Boards, we can swiftly and simply start tracking tasks, features and bugs of our project.

     

    For every project we have to create on sprint, go to Boards and select Work Items to create a new user story and provide the acceptance criteria. Select Boards and create New Work Items, then under Epic select User Story and click on Task and provide the task description.

    Setting Up Branches, Version Control and Branch Policies

     Steps:

    • Import the code from VCS.
    • Under Repos, create branches such as dev or test.
    • Set up the branches policies to improve code quality.

    Improve Code Quality With Branch Policies

    Branch policies help teams protect their important branches of development. Branch policies enforce a team’s change management standards and code quality.

    Setting up Branch Policies for Review Code

    Code reviews are good practice for the most successful software development projects. We have to configure the minimum number of reviewers to review the code and complete the pull request.

    • The code review done in a pull request is simply to find obvious bugs.
    • Code reviews help protect the team from bad commits or merges and broken builds that hinder team’s productivity.

    The review will help to catch these kind of problems before the merge, protecting our important branches, such as the release branch, from undesirable changes.

    Pre-Commit, Enabling Pull Request

    Steps:

    Workflow to complete the pull request:

    • Create a pull request
    • Complete a pull request

    Benefits of pull request:

    • Peer review
    • Enough testing and better stability
    • Reducing conflicts
    • Continuous delivery
    • Clearer responsibility

    Create a pull request:

    • Pull requests combine the review and merge the code into a single collaborative process.
    • For every bug fix or new feature in a branch, we have to create a new pull request.
    • Add the members of the team to the pull request once the development branch build is completed without issues. Then members can review and vote on your changes.

    Link work items to your pull request:

    Select the Overview tab in Azure DevOps. Select the add link button “+” in the Work Items to relate your current pull request.

     

    Complete Pull Request:

    • The basic policy requires that some reviewers approve the code with no rejections.
    • If Allow users to approve their own changes is not selected, the creator of the pull request can still vote Approve on their pull request, but their vote won’t count toward the Require, least reviewers policy.
    • If any reviewer rejects the changes, the pull request cannot be completed unless the allow users selection is completed, even if some reviewers select Waiting or Reject.
    • To reset code reviewer votes when new changes are pushed to the source branch, check Reset code reviewer votes when there are new changes.

    Nightly CI Build

    Steps:

    Build Triggers:

    • We can configure the build triggers to generate war package for every code commit made to the branch.
    • Under Build select Triggers to enable CI and include master branch.
    • One can enable the build trigger for every commit made to the branch.

    • Configure two builds—one for dev branch and one for master branch.
    • If the developer executes the code, the dev branch build is triggered automatically with help of dev branch polices.
    • Once the dev branch build is completed, reviewers approve the pull request and the master branch build is triggered. Once the build is completed, it moves to the release part.
    • The code quality analysis and unit test/Junit test is configured in the dev branch. The code quality gateways such as code review and find bug have been enabled.

    Continuous Deployment (Release) through Azure Kubernetes Services (AKS)

    Pre-requisites:

    • Any source code management tool such as Git/SVN or Azure Repo
    • Microsoft MSDN account
    • Azure account creation and VSTS/Azure DevOps account creation
    • Dockerfile to be created and placed along with source code in Git
    • YAML file to be created and placed along with source code in Git
    • Kubernetes cluster to be created in Azure portal
    • Docker hub account or Azure Container Registry
    • Build definition–Docker build image

    Docker File Creation for Application to Deploy

    Create the Service Principle to provide the access to the application/Docker image to deploy on Azure Kubernetes services. Select the Deploy to Kubernetes task and make the following configurations:

    • Create new Kubernetes service connection
    • Select the command option for deployment
    • Provide the configuration file location i.e, Deployment.yaml
    • Select the container registry type
    • Select the subscription
    • Select the Replace Tokens task to append the tag number for an image automatically while pushing it to container registry
    • Perform the deployment and view the logs

     

    Test plans:

    Load testing:

    Pipeline History

    Azure Pipelines is used to automatically build and test your code project and make it available to other users. Azure Pipelines integrates CI and CD to continuously test and build your project and ship it to any target server. It provides the full-stack end-to-end flow of CI and CD through various tools integrated as tasks in Azure DevOps.

    Conclusion

    With Azure DevOps, you can set up end-to-end DevOps with tracebility, pre-commit, CI and deployment to Kubernetes.

     

    This article was co-authored by Srinivasa Rao Guntupalli, lead engineer at HCL Technologies Ltd. He works on various DevOps technologies such as Git, maven, Jenkins and cloud technologies including AWS, Azure, Azure DevOps/VSTS.

     

    — Naveen Manchem