Tag: Azure pipelines

  • DevSecOps in Azure

    DevSecOps in Azure

    DevSecOps is everywhere, and chances are your organization is shifting toward this convergence of development, security and operations. In an on-premises environment, you build your own DevSecOps process by mixing and matching existing and new tools. However, in the cloud, much of the environment is determined by your cloud provider. Microsoft’s Azure cloud is known for its robust security capabilities. Specifically, Microsoft has developed a set of capabilities and services that can help you ease into a DevSecOps workflow without much effort. The tools are pre-integrated and support everything from CI/CD to identity management. Let’s see how they work.

    What is DevSecOps?

    Security is an important part of DevOps. But it can be challenging to ensure that software developed in a CI/CD process is really secure.

    DevSecOps is an organizational pattern that can make this possible, even at high development velocity. It is an approach the DevOps team can use to prevent security issues from the outset of the development life cycle, ensuring their software is not only high quality but also secure and resilient to failure.

    DevSecOps is a discipline that combines application development, security, operations and infrastructure-as-code (IaC) with an automated continuous delivery cycle. Here are three practices that can help a team move towards a DevSecOps process:

    • Focus on average detection and recovery times—These metrics show how long it took to detect a breach and how long it took to recover. This can be tracked through continuous field testing. Improving these indicators should be an important consideration when evaluating potential policies.
    • Defense in depth—Every component in the architecture should be secure so that if and when attackers manage to compromise the system, there are multiple security measures and controls holding them back. This puts an end to the notion of a ‘secure perimeter’ that the organization must defend at all costs. Each component must have its own secure micro-perimeter—this is a zero-trust approach.
    • Continuous learning—Teams should conduct regular assessments of security practices and their environment, especially after security incidents. Every time a security incident is discovered and resolved, the team must evaluate what went wrong in the development life cycle, explore how it enabled the incident and identify ways to improve the process.

    Features and Services That Enable DevSecOps in Azure

    The Azure cloud provides several built-in capabilities that make it possible to adopt a DevSecOps process.

    1. Azure DevOps

    Azure DevOps provides developer services that enable teams to plan tasks, collaborate on code development and build and deploy applications. Azure DevOps supports a collaborative culture and process in which developers, project managers and contributors work together to develop software. You can use Azure DevOps Services in the cloud or Azure DevOps Server on-premises.

    2. Build and Deploy Containers With Azure Pipelines

    You can integrate Azure Pipelines, the Azure cloud’s CI/CD solution, with your Kubernetes clusters. Use the same YAML document to build multi-stage CI/CD pipelines.

    Azure Pipelines lets you track metadata, such as commit hashes and issue numbers, from Azure Boards into container images. This provides direct traceability from any security issue to a specific change made in development. It also provides clear, easy-to-read documentation that can improve the feedback loop between development, operations and security teams.

    3. Run and Debug Containers with Dev Spaces

    When developing Kubernetes applications, there is a need to test applications locally and understand how they interact with dependent services. You may need to develop and test multiple services in collaboration with other developers or teams.

    Azure provides Bridge to Kubernetes, which lets you run and debug code on your development machine while connecting to your Kubernetes cluster. You can test your code end-to-end, set breakpoints on code running on the cluster and share your development cluster among team members. This makes it possible to test and resolve Kubernetes security issues in a realistic environment before deploying to production.

    4. Manage Identities and Access with Azure AD

    The Microsoft Identity Platform takes the Azure Active Directory (Azure AD) developer platform one step further. It allows applications to accept logins from any Microsoft identity, and obtain tokens that can be used to call Microsoft APIs or APIs created by other developers. This creates a large, interoperable ecosystem.

    Azure Active Directory B2C provides B2C identity services—customers get single sign-on (SSO) access to applications and APIs using their preferred social, enterprise or local account ID. Another option is to integrate Azure AD with on-premises Active Directory for hybrid and Azure migration scenarios.

    You can also use Azure Role-Based Access Control (RBAC) to manage access to cloud resources. RBAC lets you manage who can access your Azure resources, what they can do with them, and which regions can access them.

    You can also use the Microsoft Identity Platform to protect DevOps tools themselves, including native support for Azure DevOps and integration with GitHub Enterprise.

    5. Manage Keys and Secrets with Azure Key Vault

    Exposed secrets are a severe security issue that is very common in modern applications. Azure Key Vault lets you manage the distribution of secrets by storing them centrally in Azure Key Vault. Key Vault greatly reduces the chance of accidentally disclosing secrets. With Key Vault, application developers no longer need to store credentials or other sensitive information in their application code.

    6. Azure Policy and Azure Security Center

    Azure Policy lets you specify a default allowed configuration that is automatically applied to all cloud resources. This can avoid misconfigurations that violate security policies. Azure Policy works on the basis of desired state configuration (also known as declarative configuration), letting you specify to what degree resources and services should be secured and whether to alert or block/modify deployments in Azure if they don’t meet the policy.

    When using Azure in multi-tenant mode, policies can be applied at the management group, subscription or resource group level. You can define specific policies and enforce compliance requirements for test, staging, and production environments.

    The Azure Security Center (ASC) and Advisor can also be easily integrated into the DevOps process. ASC makes it possible to augment event streams in Azure with threat intelligence or analysis by third-party security tools.

    7. Penetration Testing

    Penetration testing is the recommended method for examining your environment for infrastructure or application configuration vulnerabilities that could create vulnerabilities that an attacker could exploit.

    An Azure penetration test should check for vulnerabilities on endpoints, apply fuzzing (malformed input) to discover business logic vulnerabilities, and perform port scanning to identify network vulnerabilities.

    Microsoft provides specific guidance for penetration testing in Azure with recommended products and penetration testing service providers.

    8. Configuration and Infrastructure Scanning

    To ensure security for cloud subscriptions and resource configuration across multiple subscriptions, you can use the tenant security solution part of the Secure DevOps Kit for Azure (AzSK).

    In addition, you can leverage Microsoft security technologies like Microsoft Defender for Cloud and Microsoft Sentinel. These solutions provide monitoring and security features designed to detect and alert on unusual events or configurations that require investigation and possible remediation.

    Conclusion

    In this article, I introduced DevSecOps and showed how the Azure cloud provides a range of services that let you easily implement DevSecOps practices. These include:

    • Azure DevOps: The venerable toolset with strong support for DevSecOps environments.
    • Azure Pipelines: The home-grown CI/CD product.
    • Azure AD: A comprehensive zero-trust identity management solution.
    • Azure Policy and Security Center: Enables unified control over security policies for cloud automation and security visibility over cloud workloads.

    I hope this will be useful as you take your first steps to practice DevSecOps in the cloud.

  • Pre-Commit Job Workflow in Azure DevOps

    Pre-Commit Job Workflow in Azure DevOps

    Many organizations use a combination of open so Many organizations use a combination of open source and commercial tools and Azure DevOps (formerly called VSTS) and an application lifecycle management (ALM) tool that includes features such as Azure pipelines. This article talks about how to achieve an end-to-end DevOps setup in Azure

    Pre-requisites:

    1. Subscription
    2. Project created with version control selected
    3. Process template selected (Agile/Scrum)
    4. Code repo–one on VSTS and one on GitHub/cloud GitLab
    5. Build server–Use credits from MSDN subscription
    6. Deploy–Azure subscription from MSDN
    7. Other tools–tools supported by VSTS/Azure DevOps
    • The objective of a pre-commit job is to reduce the risk of complexities, duplications and potential bugs in the code to get deployed directly into production.
    • In usual workflows, the build will be generated on regular intervals and released for testing. However, there is a chance of several code quality issues and building deployment issues.
    • With the help of pre-commit job, we can configure triggers to generate builds for every pull request made by developers. Also, this enables the development team to create pull request releases that deploy your pull request code or pull request builds, which help to detect deployment/release issues before the code changes are merged.
    • Pre-commit deployments also help catch deployment/release issues early in the cycle, maintain better code quality and release with higher confidence.

    Configuring Agile Board in Azure DevOps for Project Management, and Tracking Steps to Do It

    Steps:

    With Azure Boards we can plan, track and discuss work across different teams. With Azure Boards, we can swiftly and simply start tracking tasks, features and bugs of our project.

     

    For every project we have to create on sprint, go to Boards and select Work Items to create a new user story and provide the acceptance criteria. Select Boards and create New Work Items, then under Epic select User Story and click on Task and provide the task description.

    Setting Up Branches, Version Control and Branch Policies

     Steps:

    • Import the code from VCS.
    • Under Repos, create branches such as dev or test.
    • Set up the branches policies to improve code quality.

    Improve Code Quality With Branch Policies

    Branch policies help teams protect their important branches of development. Branch policies enforce a team’s change management standards and code quality.

    Setting up Branch Policies for Review Code

    Code reviews are good practice for the most successful software development projects. We have to configure the minimum number of reviewers to review the code and complete the pull request.

    • The code review done in a pull request is simply to find obvious bugs.
    • Code reviews help protect the team from bad commits or merges and broken builds that hinder team’s productivity.

    The review will help to catch these kind of problems before the merge, protecting our important branches, such as the release branch, from undesirable changes.

    Pre-Commit, Enabling Pull Request

    Steps:

    Workflow to complete the pull request:

    • Create a pull request
    • Complete a pull request

    Benefits of pull request:

    • Peer review
    • Enough testing and better stability
    • Reducing conflicts
    • Continuous delivery
    • Clearer responsibility

    Create a pull request:

    • Pull requests combine the review and merge the code into a single collaborative process.
    • For every bug fix or new feature in a branch, we have to create a new pull request.
    • Add the members of the team to the pull request once the development branch build is completed without issues. Then members can review and vote on your changes.

    Link work items to your pull request:

    Select the Overview tab in Azure DevOps. Select the add link button “+” in the Work Items to relate your current pull request.

     

    Complete Pull Request:

    • The basic policy requires that some reviewers approve the code with no rejections.
    • If Allow users to approve their own changes is not selected, the creator of the pull request can still vote Approve on their pull request, but their vote won’t count toward the Require, least reviewers policy.
    • If any reviewer rejects the changes, the pull request cannot be completed unless the allow users selection is completed, even if some reviewers select Waiting or Reject.
    • To reset code reviewer votes when new changes are pushed to the source branch, check Reset code reviewer votes when there are new changes.

    Nightly CI Build

    Steps:

    Build Triggers:

    • We can configure the build triggers to generate war package for every code commit made to the branch.
    • Under Build select Triggers to enable CI and include master branch.
    • One can enable the build trigger for every commit made to the branch.

    • Configure two builds—one for dev branch and one for master branch.
    • If the developer executes the code, the dev branch build is triggered automatically with help of dev branch polices.
    • Once the dev branch build is completed, reviewers approve the pull request and the master branch build is triggered. Once the build is completed, it moves to the release part.
    • The code quality analysis and unit test/Junit test is configured in the dev branch. The code quality gateways such as code review and find bug have been enabled.

    Continuous Deployment (Release) through Azure Kubernetes Services (AKS)

    Pre-requisites:

    • Any source code management tool such as Git/SVN or Azure Repo
    • Microsoft MSDN account
    • Azure account creation and VSTS/Azure DevOps account creation
    • Dockerfile to be created and placed along with source code in Git
    • YAML file to be created and placed along with source code in Git
    • Kubernetes cluster to be created in Azure portal
    • Docker hub account or Azure Container Registry
    • Build definition–Docker build image

    Docker File Creation for Application to Deploy

    Create the Service Principle to provide the access to the application/Docker image to deploy on Azure Kubernetes services. Select the Deploy to Kubernetes task and make the following configurations:

    • Create new Kubernetes service connection
    • Select the command option for deployment
    • Provide the configuration file location i.e, Deployment.yaml
    • Select the container registry type
    • Select the subscription
    • Select the Replace Tokens task to append the tag number for an image automatically while pushing it to container registry
    • Perform the deployment and view the logs

     

    Test plans:

    Load testing:

    Pipeline History

    Azure Pipelines is used to automatically build and test your code project and make it available to other users. Azure Pipelines integrates CI and CD to continuously test and build your project and ship it to any target server. It provides the full-stack end-to-end flow of CI and CD through various tools integrated as tasks in Azure DevOps.

    Conclusion

    With Azure DevOps, you can set up end-to-end DevOps with tracebility, pre-commit, CI and deployment to Kubernetes.

     

    This article was co-authored by Srinivasa Rao Guntupalli, lead engineer at HCL Technologies Ltd. He works on various DevOps technologies such as Git, maven, Jenkins and cloud technologies including AWS, Azure, Azure DevOps/VSTS.

     

    — Naveen Manchem