devopsy.kinsta.cloud

Category: DevOps Practice

  • Scalability in the Age of DevOps: A Must for Success

    Scalability in the Age of DevOps: A Must for Success

    DevOps seems to be an emerging topic “du jour” in the world of software development. And like it or not, it is showing no signs of slowing down. If used correctly, DevOps can be helpful—from making teams (developers and testers) work seamlessly to automating routine tasks, DevOps has proved its awesomeness in just a few years. So what is helping make DevOps a huge success? Scalability!

    Technically speaking, scalability is something that defines the exact capability of the system, and it has to increase frequently to accommodate the ever-changing needs of the world. As customers rely more heavily on your services, achieving hyper-scale is becoming more crucial. To ensure scalability, you may need to redesign or rethink, your different processes and systems.

    Say, for instance, customer demand seems to have reached its apex. Preparing comprehensive strategies that focus on scalability has become prime importance.

    To get the most out of your application, invest in DevOps-related time and dollars. With the introduction of the cloud, developers are focusing on building features and products while ops (operations) are making sure everything stays in sync. At the same time, many traditional ops functions have been outsourced to cloud providers.

    Scaling DevOps    

    Here are some best practices that can assist you in creating scalable, secure and high-performance deployments.

    Be Vigilant

    With the increase in the adoption of visualization, networks are more complex and heterogeneous, and so are the security risks associated with them. The only way left to manage these networks is by automating key management functions. Here is when DevOps comes into play. But for that, the method must be included with security as a key component, or the volume and pace of network would eventually change—several technologies such as SDN and virtualization will skyrocket the level of IT risk like never before. Another challenge is that security is still considered as an afterthought. And so-called business inhibitors are seen advising an organization on what can’t be done instead of how to do things securely. This is an area where security, developers and operation teams must foster a level of trust, or collaboration won’t exist. The only way to do this is incrementally and with vigilance.

    Watch Changes Carefully

    Considering DevOps as a collaborative mindset enforces professionals to come up with faster and more efficient ways to deploy, operate and upgrade applications. When new releases are delivered, several tools can ensure consistent configuration of infrastructure and software components. These tools also can be used to remediate security concerns by constantly validating the proper application of security’s best practices and taking automatic countermeasures.

    Ease of Use

    Automation and repetitiveness are what DevOps is all about. Fortunately, there are a few tools that are capable of automating the building and configuration of VMs. However, make sure you have the adequate horsepower to run this capability, either in-house—which is trickier to scale dynamically—or in the cloud, if your product is amenable to that. At the end of the day, it’s all about how you make your product installation and configuration easy.

    Managing Gateways

    While building any new goal or coming up with the best culture between development and operation teams, keep certain gates between functions to ensure that the production environment remains stable. Try to structure it in such a way that operations include development discussions and daily scrums. By doing this, operations teams simultaneously will understand what changes will be made in future releases. By integrating automation in both testing and operations, you will be able to find issues before they reach production. In addition, you’ll realize cost efficiencies and better quality operations.

    Conclusion

    In today’s fast-paced development environment, a well-thought preparation strategy for mass scalability is a must to make the entire process smooth and easy. DevOps reduces lead-up anxiety by getting a head start on planning, so you have more time to enjoy the fruits of your labor. Adding scalability in your strategy enables you to experiment and try new things without fearing failure. Testing new approaches ahead of time to see what does or doesn’t work will only help you learn and make your scalability efforts that much more effective for the future, as your company continues down the path of hyper-growth and hyper-scale.

    — Herman Morgan

  • DevOps Tools Democratize the View From Both Sides

    DevOps Tools Democratize the View From Both Sides

    How modern product decisions platforms marry the key metrics of features changes to organizational KPIs

    In 1967, Joni Mitchell released one of her most iconic songs, “Both Sides Now.” The song was inspired by the book, “Henderson the Rain King,” in which author Saul Bellow writes about flying to Africa and looking out the plane window, “Look at those clouds down there; it’s very strange to look at the clouds from up above them. I remember as a boy having dreamed up at clouds a lot and having seen the cloud from both sides now, I suppose I shouldn’t really be amazed by anything.”

    Although not her intention and slightly before its time of popularity, the song speaks quite accurately to the tensions between idealized DevOps and the reality on the ground that many organizations are struggling through. On paper, the core tenant of DevOps is about cooperation, seeing problems and processes from multiple team members’ perspectives from each side, but this is easier said than done.

    “But now old friends they’re acting strange
    They shake their heads, they say I’ve changed
    Well something’s lost, but something’s gained
    In living every day.”

    The Unicorns and the Rest of Us

    Some of today’s most successful and disruptive companies—including Google, Netflix and Facebook—have taken their market-leading positions precisely because they have developed strategy and culture around DevOps. It has been part of the company DNA since the beginning and every employee is bought into it. This means fast, agile development and experimentation is part of the culture.

    While many companies would like to capture some of the benefits that unicorns have realized from implementing these development methodologies, for many, this simply is not an option. There is legacy infrastructure, processes and people—who are sometimes the hardest to change. Often DevOps is embraced in small pockets of a company, by teams with newer members working on new projects. The mindset shift that needs to happen to have DevOps—and everything that comes along with it—become part of a company’s culture takes time, commitment and perseverance. However, a good way to start is by demonstrating the fresh, diverse perspectives that can be gained through experimentation and progressive exposure to encourage wider adoption.

    Leveling Up

    One of the ways modern app delivery, operations and business management teams at organizations all along the adoption spectrum are achieving governance, operational awareness and control at speed is by wrapping new code deployments in feature flags and using a centralized platform to control progressive exposure to users.

    By themselves, feature flags are not new. What is new is the centralized and externalized control plane as a service and, newer still, the idea of observing system and business KPIs at the user session or request granularity level for feature flag-controlled decisions.

    When something is rolled out to 5% of the users, it is not likely to light up global ops or business metrics dashboards, even if it is a bit broken—and that makes people cranky. If instead, tools are in place to monitor the KPIs within that 5% population and compare it to the other 95% looking for divergence, you have a much more powerful early warning system.

    These new centralized platforms also extend observability to stakeholders beyond the few wizards driving orchestration and ops teams watching dashboards. An anti-pattern to the local control autonomy spiel modern DevOps patterns call for, this wider visibility protects against local optimization traps in which one team improves its KPI while degrading other KPIs it doesn’t know or care about.

    It can be beneficial to have the idea of key metrics tied to a feature and organizational metrics that look after larger organizational concerns. This inspiration can be traced back to what LinkedIn does with its XLNT experimentation platform, where it calls the organizational metrics “guardrail KPIs.” By always tracking a set of metrics that are critical to the business as a whole, organizations gain a clearer view of the impact that both the granular updates and the larger rollouts are making to the business.

    As Joni said in all her wisdom:

    “I’ve looked at life from both sides now
    From up and down, and still somehow
    It’s life’s illusions I recall
    I really don’t know life at all.”

    No team or organization can know everything all the time, but we can build in visibility to the process of rolling out functionality to our users and create a shared vision of our intentions and the results we achieve. I would argue that if you do look at DevOps from both sides, up and down, you will know the benefits.

    — Dave Karow

  • Successful Digital Transformation Starts With Developers

    Successful Digital Transformation Starts With Developers

    Digital transformation (DX) continues to be a primary focus as companies both large and small look to become more agile, improve and accelerate the delivery of product and service offerings and enhance data-driven business decisions. The C-suite typically leans on IT departments and developers to champion such transformations, designing the software and tools that will lead the experience both internally with staff and externally with customers. The challenge often lies in the fact that most organizations lack the software resources to drive a successful transformation. In addition, research has found that 45% of executives don’t know where to start with DX projects, resulting in added pressure on the developers who hold the technical knowledge but may lack the managerial expertise for a successful rollout.

    So, with IT having the opportunity to be the hero or the scapegoat of a DX initiative, where should development teams start? Typically, digital transformation requires development transformation, so you can start at home, by modernizing the way you develop and deliver software. Doing this also significantly improves the odds of a successful digital transformation and puts development in the driver’s seat.

    Here are four factors to prioritize before kicking off a DX project:

    Buy vs. Build

    A DX initiative requires a number of digital tools and software applications, some of which can be purchased through third parties while others may need to be built in-house, depending on the desired goals of the program. Organizations must determine whether they want to build or buy new software solutions, and the answer is almost always mixed. Most developers believe that each application can be built in-house, while the C-suite often encourages purchases from a third party to accelerate adoption. Before starting to code, ask yourself if the solution you need should be bought or built. Oftentimes, the same goal can be achieved through a purchased solution, empowering you and your team to spend more time customizing those solutions rather than starting from scratch while focusing on the projects that will have the biggest impact on the business—ultimately accelerating time to completion.

    Accelerated Rate of Change

    Once applications are built and available to the public, you can expect bugs to be identified and requests for improvements to be made at an accelerated rate. Developers are constantly making software updates, but when applications are being used to support DX, the rate of change for business requirements is going to speed up, especially when those applications are customer-facing. Developers will need to learn fast and iterate quickly, so it’s important to establish an agile culture within your team.

    Adopting a DevOps philosophy often will be the most successful approach, as it enables your team to test and troubleshoot quickly and enact change across the platform. This approach also assists the rest of the organization in adopting DX practices, delivering a more seamless experience and universal data-sharing. Developer teams must be wary of traditional quarterly or annual planning cycles that create silos, slow agile processes and hinder successful implementations. By breaking down these silos, your team will gain better insight into what is and isn’t working and enable swifter improvements, creating a culture that embraces the changes that come with DX.

    Modular Software Architecture

    Software architecture is incredibly important, as it directly impacts the team’s ability to react to rapid change. A more loosely coupled, modular architecture puts the developer team in the best position to evolve quickly with fewer dependencies, enabling them to make updates and replace components over time to meet business objectives and customer demands. The ability to reconfigure or replace components without having to recode the entire system also assists in build-versus-buy decisions, with developer teams purchasing add-ons to incorporate into a system and solve problems quickly.

    DX Outside of Organizational Barriers

    We work and collaborate in a digital ecosystem. External APIs, third-party data-sharing and interoperability are all requirements as organizations roll out new tools as part of a DX initiative. DX comes down to building a digital ecosystem that flows outside of the organization, whether it is employees working remotely or customers integrating your tools into their existing solutions. Your company’s software needs to accommodate these capabilities to enable a more seamless and well-rounded DX approach.

    DX is a Journey, Not a Destination

    A successful DX initiative should be seen as a culture shift, not an end result. The digital world is not static, and DX is a practice that enables an organization to learn fast and be lean and nimble. As soon as you’ve reached one milestone, there’s a whole new set of rules to play by to get to the next, requiring the developer team to recalibrate. By taking a realistic, process-oriented approach, the developer team will have the mindset and tools necessary to make DX successful.

    — Jeff Fried

  • 5 Tips for Launching New App Features

    5 Tips for Launching New App Features

    Are you looking to improve your organization’s mobile app? For some the process may be relatively straightforward. But ensuring you cover all steps correctly will go a long way to a successful app feature release. Here are five practical tips to successfully launch a new app feature.

    Include the App Feature Release at the Start

    While you’re still in the planning phase for a new feature, it’s a good idea to think about how you will release it. This is something often done within the design process.

    Things you should incorporate into this decision include:

    • Who will see this app feature first? (Are there internal or external beta groups?)
    • How will this feature’s success be measured?
    • Who will see the app feature once it’s in a steady state? (Will the feature be only for VIP customers or for everyone?)
    • Is there important timing tied to this release, such as an event or special time of the calendar year?

    Tools that will help you with this step include product delivery and tracking tools.

    Build Awareness

    Awareness around a release is important for both internal and external groups. Within your organization, do teams have the support they need to be successful? Think about what your sales, marketing, customer success or any other team will need in terms of understanding the app feature being released, and how to answer any questions they might face. Externally, awareness should be tied back to how you will measure success.

    Tools that will help you accomplish this include go-to-market plans, centralized information repositories and any other tools that will help your teams (and customers) stay connected, informed and collaborative.

    Measure the Release of the App Feature

    After the release has happened, how will you know if it was successful? Because you already thought about success metrics in the planning stage, you should be ready to measure whether the feature was successful or not.

    Tools that will help with this include those that surface sales and ops metrics. Also, it’s important to consider these together—look at performance and monitoring metrics, support requests by volume and qualitative feedback from customers and prospects.

    Celebrate and Recognize

    Take time to celebrate your wins. Shipping software is like exercising a muscle: The more frequently you do it, the easier it is to execute. If you ship less frequently, the process begins to atrophy and the action becomes more difficult. Celebration (even for small wins) motivates to continue practicing the act of shipping and results in more stable services and products.

    Reflect and Iterate

    Software is never done, and neither is a process for software delivery. After the release has occurred and you’ve paused to enjoy the moment, now it’s time to reflect on what went well and what didn’t. Reflect on both process and product. Tie process back to culture—consider the tools that you use for the process, what enabled you to do more and what was a hindrance. Use this feedback and apply what you learned from measuring success in the planning phase for the next release. Learn how you can adjust and improve upon what you shipped.

    — Adam Zimman

  • Continuous Reliability: How QA and SREs Can Improve Their CI/CD Workflow

    Continuous Reliability: How QA and SREs Can Improve Their CI/CD Workflow

    A key challenge in maintaining and accelerating the speed of software delivery is balancing the rate of change with the reliability of your software. With the move toward an increasingly automated workflow, as well as teams of dozens to hundreds of engineers introducing changes at a high frequency, the ability to identify the issues that will have a severe impact early on becomes critical.

    This challenge poses many questions: With so many errors, will spending my precious time to resolve a particular issue be really worth it? How do I detect severe issues in code or infrastructure that could be impactful ahead of time? Once a release has been moved forward, how can I know how well it’s doing–should it be fully deployed, hotfixed or even rolled back?

    That’s why we need our CI/CD workflow to work both ways, helping us upgrade and improve our application as well as allowing us to learn and adapt according to how it performs with our users. Enter continuous reliability.

    A Feedback Loop Between Everyone

    Code quality gates and contextual feedback loops are the CI/CD building blocks that define the emerging practice of continuous reliability. The current and most common feedback loop is between dev and QA teams, and it’s usually the backbone of every good development cycle. However, feedback loops are relevant to the whole development cycle, and they don’t end with QA/dev deploying and testing elements in staging environments.

    As we all know, production tends to display unexpected and surprise-oriented behavior, some might even say that issues and bugs appear out of thin air. That’s why QA and SRE teams need accurate answers as to the actual quality of releases. They need to identify when critical anomalies move from staging and production, and ideally have the ability to gate them before they harm the application, workflow or customers.

    Defining the Anomalies That Matter

    Anomaly detection is critical to continuous reliability. In most applications, error and performance anomalies can be mapped into three core types:

    1. New defects: the introduction of new errors into a system, usually a result of code changes.
    2. Increasing Errors: an increase in the relative volume (i.e. as compared to throughput) of certain errors, usually as a result of a code or infrastructure change.
    3. Slowdowns: a decrease in the response time of critical parts of the application due to inefficient code or improper provisioning or configuration of infrastructure.

    The introduction of any of these can lead to CPU, GC and IO increases. As such, it is not only imperative that QA and SRE teams gate them from moving into production, but also to prioritize and assign severities that help tackle the most important issues.

    However, detecting anomalies in large-scale systems can be nearly impossible due to a high noise-to-signal ratio, and we’re often left to rely on our users and customers to alert us when something goes wrong. That’s why there’s a need for an objective measure to quantify the quality of a release—one that lies in being able to converge machine learning and critical code-level data to automatically gate bad releases from moving up the chain.

    Continuous Reliability Through Quality Gates

    Once we have an understanding as to what should count as an anomaly within our application, the next step is setting a quality gate to help us identify, understand and hopefully stop bad code from being promoted to production.

    Taking the three core type of anomalies—new issues, regressions and slowdowns—we can evaluate four core quality gates that our code should pass before being promoted:

    1. Error Volume: The normalized error rate of an application, based on calls into the code, should never increase between releases.
    2. Unique Error Count: The number of unique error counts, especially in key applications, should not increase between releases.
    3. New Errors: New errors of a critical type or occurring at a high rate should block a release.
    4. Increasing Errors and Slowdowns: Severe regressions and slowdowns should block a release.

    These four gates can help you define a benchmark that is powerful and broadly applicable to complex environments, but also isn’t complex to the point that it remains an academic exercise. They will help you understand whether your code is ready to be deployed to your pre-production or production environment, and help reduce or even eliminate errors, issues, slowdowns and anomalies within your application.

    — Tal Weiss

  • Getting the Enterprise Digital Core in Shape

    Getting the Enterprise Digital Core in Shape

    While enterprises rush to transform their business digitally, most are taking tactical steps rather than building a comprehensive digital core that can be improved over time. Very few are taking a thorough view of their entire technology portfolio and looking for ways to create a cohesive core—their digital core—to provide them the strength and stability to move forward swiftly in their digitization efforts. The enterprise digital core essentially is the vital set of technologies and employee skill sets that enable the organization to deliver its digital services. These services are both consumed by customers and internally by staff and others.

    How do enterprises get their digital core into shape?

    Emphasize Talent Over Technology

    While a strong digital core includes the technological platform, it is also a people-platform. “The heart of digital transformation isn’t technology but the mindset and skill sets of your staff,” said Andy Lin, senior vice-president of strategy and consulting at Nexient. Digitization, he said, means enabling customers to transact and interact with you anytime they want, from anywhere, using any device. The only way to get there is if leadership makes it clear to staff why digitization is crucial; otherwise, they will struggle to adapt, he said.

    If the staff does understand the reasons behind the transformation, they will—instead of just following rules for the sake of rule-following—be much more likely to adopt a digital mindset. They’ll also be more likely to embrace the training and skill set development they’ll need to make the transformation a success, Lin added.

    Scrum.org CEO and product owner Dave West agreed that the digital core starts with the customer. “There needs to be a clear line from the mission and strategy of the business with the outcomes the customer wants,” he said. Teams need to understand clearly how their work helps deliver the organizational mission digitally to the customer, no matter what that mission may be. That means everyone needs to be clear on the nature of the business, or organizational business, as well as what success looks like

    West also advised organizations that seek to improve their digital core to look both inward and outward for support. From within the organization, of course, support should come from leadership, who must set the course and ensure efforts are going in the right direction. “Externally you need to build a network of support from people and organizations that have done it before,” he said.

    Put Cloud at the Core

    Of course, a strong digital core must have a strong technological foundation. In today’s typical enterprise, that may very likely mean more about subtraction than addition. While emerging technologies, such as internet of things (IoT), artificial intelligence (AI)/machine learning (ML) and others are recommended to be integrated within enterprise software, “Many of them are not generally suited to run on legacy IT infrastructure,” said Maxim Ivanov, CEO at enterprise software development firm Aimprosoft. “The digital core relies on intelligent automation scalable platforms that are integrated in the cloud and prepares data for next-generation technologies.”

    Gabriel Salazar, digital solutions manager at systems integrator Enstoa, agreed that building a cloud-based infrastructure goes a long way to improving the digital core. “It’s important, however, not to simply carbon-copy the business processes used prior,” said Salazar. “Look to take advantage of all automation, machine learning and analytics offered by converting and folding them into your updated business process.”

    Finally, if organizations want to have a strong digital core, they are going to have to work to maintain it. “Due to the speed of technology change, getting your digital core in shape is a continuous process. Start by understanding what is preventing you from supporting business goals, what is causing ongoing stability or performance issues and what reduces your time to market,” said Nick Polachek, senior principal of strategy and transformation at Maven Wave Partners.

    To do so, Polanchek recommended building the digital core iteratively while maintaining the focus on delivering business value or resolving current issues. It’s also important, he said, to have a thoughtful future vision for your digital core and ensure that each delivery is at least aligned, if not an improvement, over core technology already in place.

    After all, keeping that digital core in shape is an ongoing process that will require ongoing work and practice.

    — George V. Hulme

  • Pre-Commit Job Workflow in Azure DevOps

    Pre-Commit Job Workflow in Azure DevOps

    Many organizations use a combination of open so Many organizations use a combination of open source and commercial tools and Azure DevOps (formerly called VSTS) and an application lifecycle management (ALM) tool that includes features such as Azure pipelines. This article talks about how to achieve an end-to-end DevOps setup in Azure

    Pre-requisites:

    1. Subscription
    2. Project created with version control selected
    3. Process template selected (Agile/Scrum)
    4. Code repo–one on VSTS and one on GitHub/cloud GitLab
    5. Build server–Use credits from MSDN subscription
    6. Deploy–Azure subscription from MSDN
    7. Other tools–tools supported by VSTS/Azure DevOps
    • The objective of a pre-commit job is to reduce the risk of complexities, duplications and potential bugs in the code to get deployed directly into production.
    • In usual workflows, the build will be generated on regular intervals and released for testing. However, there is a chance of several code quality issues and building deployment issues.
    • With the help of pre-commit job, we can configure triggers to generate builds for every pull request made by developers. Also, this enables the development team to create pull request releases that deploy your pull request code or pull request builds, which help to detect deployment/release issues before the code changes are merged.
    • Pre-commit deployments also help catch deployment/release issues early in the cycle, maintain better code quality and release with higher confidence.

    Configuring Agile Board in Azure DevOps for Project Management, and Tracking Steps to Do It

    Steps:

    With Azure Boards we can plan, track and discuss work across different teams. With Azure Boards, we can swiftly and simply start tracking tasks, features and bugs of our project.

     

    For every project we have to create on sprint, go to Boards and select Work Items to create a new user story and provide the acceptance criteria. Select Boards and create New Work Items, then under Epic select User Story and click on Task and provide the task description.

    Setting Up Branches, Version Control and Branch Policies

     Steps:

    • Import the code from VCS.
    • Under Repos, create branches such as dev or test.
    • Set up the branches policies to improve code quality.

    Improve Code Quality With Branch Policies

    Branch policies help teams protect their important branches of development. Branch policies enforce a team’s change management standards and code quality.

    Setting up Branch Policies for Review Code

    Code reviews are good practice for the most successful software development projects. We have to configure the minimum number of reviewers to review the code and complete the pull request.

    • The code review done in a pull request is simply to find obvious bugs.
    • Code reviews help protect the team from bad commits or merges and broken builds that hinder team’s productivity.

    The review will help to catch these kind of problems before the merge, protecting our important branches, such as the release branch, from undesirable changes.

    Pre-Commit, Enabling Pull Request

    Steps:

    Workflow to complete the pull request:

    • Create a pull request
    • Complete a pull request

    Benefits of pull request:

    • Peer review
    • Enough testing and better stability
    • Reducing conflicts
    • Continuous delivery
    • Clearer responsibility

    Create a pull request:

    • Pull requests combine the review and merge the code into a single collaborative process.
    • For every bug fix or new feature in a branch, we have to create a new pull request.
    • Add the members of the team to the pull request once the development branch build is completed without issues. Then members can review and vote on your changes.

    Link work items to your pull request:

    Select the Overview tab in Azure DevOps. Select the add link button “+” in the Work Items to relate your current pull request.

     

    Complete Pull Request:

    • The basic policy requires that some reviewers approve the code with no rejections.
    • If Allow users to approve their own changes is not selected, the creator of the pull request can still vote Approve on their pull request, but their vote won’t count toward the Require, least reviewers policy.
    • If any reviewer rejects the changes, the pull request cannot be completed unless the allow users selection is completed, even if some reviewers select Waiting or Reject.
    • To reset code reviewer votes when new changes are pushed to the source branch, check Reset code reviewer votes when there are new changes.

    Nightly CI Build

    Steps:

    Build Triggers:

    • We can configure the build triggers to generate war package for every code commit made to the branch.
    • Under Build select Triggers to enable CI and include master branch.
    • One can enable the build trigger for every commit made to the branch.

    • Configure two builds—one for dev branch and one for master branch.
    • If the developer executes the code, the dev branch build is triggered automatically with help of dev branch polices.
    • Once the dev branch build is completed, reviewers approve the pull request and the master branch build is triggered. Once the build is completed, it moves to the release part.
    • The code quality analysis and unit test/Junit test is configured in the dev branch. The code quality gateways such as code review and find bug have been enabled.

    Continuous Deployment (Release) through Azure Kubernetes Services (AKS)

    Pre-requisites:

    • Any source code management tool such as Git/SVN or Azure Repo
    • Microsoft MSDN account
    • Azure account creation and VSTS/Azure DevOps account creation
    • Dockerfile to be created and placed along with source code in Git
    • YAML file to be created and placed along with source code in Git
    • Kubernetes cluster to be created in Azure portal
    • Docker hub account or Azure Container Registry
    • Build definition–Docker build image

    Docker File Creation for Application to Deploy

    Create the Service Principle to provide the access to the application/Docker image to deploy on Azure Kubernetes services. Select the Deploy to Kubernetes task and make the following configurations:

    • Create new Kubernetes service connection
    • Select the command option for deployment
    • Provide the configuration file location i.e, Deployment.yaml
    • Select the container registry type
    • Select the subscription
    • Select the Replace Tokens task to append the tag number for an image automatically while pushing it to container registry
    • Perform the deployment and view the logs

     

    Test plans:

    Load testing:

    Pipeline History

    Azure Pipelines is used to automatically build and test your code project and make it available to other users. Azure Pipelines integrates CI and CD to continuously test and build your project and ship it to any target server. It provides the full-stack end-to-end flow of CI and CD through various tools integrated as tasks in Azure DevOps.

    Conclusion

    With Azure DevOps, you can set up end-to-end DevOps with tracebility, pre-commit, CI and deployment to Kubernetes.

     

    This article was co-authored by Srinivasa Rao Guntupalli, lead engineer at HCL Technologies Ltd. He works on various DevOps technologies such as Git, maven, Jenkins and cloud technologies including AWS, Azure, Azure DevOps/VSTS.

     

    — Naveen Manchem

  • Why Your Test Automation Is Failing

    Why Your Test Automation Is Failing

    If you have been investing in test automation, but it’s not getting you where you think it should, it could be that you approached it superficially.

    One of the most common issues is that the test part of test automation is rather straightforward to automate, especially with today’s abundance of great automation tools. But jumping straight into automated testing leaves you with a big heap of automation scripts, and often not the best ROI. Understanding why requires looking into the automation process.

    When an engineer builds an automated test, they do it in a test environment that they (or someone in the organization) set up. Usually, this is the same way manual testers get access to test environments.

    Before the engineers start automating, they have a working copy of the product they are testing, with relevant data on it. They have a good idea of the test they want to write. Everything looks promising.

    They write the test and give it a few runs. Everything is looking nice and stable. Now, to provide the ROI we were hoping for, the test must run a million times. Or, at least, a few thousand times, right?

    To reap the benefits of automation, it needs to happen frequently enough. However, the bottleneck is often discovered after creating a bunch of tests. Because the tests must run on a similar test environment to the one used to create them, with the new versions of the product or with any introduced changes that are being tested, you need access to hundreds of environments. This requirement makes it difficult to test effectively.

    Below are three test environment approaches that inhibit test automation.

    Manually Setting Up an Environment Each Time to Run Automated Tests

    For tests that don’t need to run as frequently, such as security testing and performance testing, it’s common to set up an environment for automated tests every time the test needs to be executed. As a result, testers must wait for the environment to be ready, perhaps do some additional configurations and then run a suite of tests. And, if they’re trying to optimize utilization, the environment is left to run overnight or weekend.

    The problem with this approach is it takes the “auto” out of automated testing. When automation depends on a manual step, the test is only as efficient as this manual step.

    If the manual step isn’t very efficient, automation may not be helping at all. I’ve seen companies take so long to get an environment to work that running automated tests on it didn’t make any difference from a time perspective.

    Setting Up a Static Environment That Automation Always Runs On

    When setting up an environment takes a long time, it’s natural to assume it’s worth leaving an environment running so test automation can run on it. The preparations were made, the infrastructure is all in place, the product is configured and the data is ready, so let the automation party begin.

    While complex, static test environments may seem ideal, but the party doesn’t last long. First, since it usually takes some time to put the environment together (same as the previous case), it often becomes high-maintenance. It can be scary to make changes that could ruin things, and you can end up needing someone to manage change full-time. Much like chocolate cake in the office kitchen, an environment that mimics production with good data on it tends to attract everyone. Soon, these nice environments become shared assets and tests start conflicting for the same resources. The natural next step is to set up another static environment. This loop usually ends with an angry manager saying something about cutting CAPEX.

    Automating the Environment and Cutting Ourselves Some Slack

    With the rise of infrastructure as code (IaC) and immutable infrastructure, there are more ways to spin up a test environment. Resourceful test teams will try to automate the environment creation or use built-in test tool or CI system capabilities to run tests on a public cloud instance or a container, if the product they test allows that. This is cheap and useful—and could work for some simple tests. The challenge usually starts with everything this approach doesn’t cover. It can be difficult to use IaC to spin up a test environment and handle all aspects of it—from security and isolation to dependencies, data, third-party integration and more. This is where this approach is often blocked and sends us back to the previous two approaches for a big part of the testing load—which again makes test automation investment less fruitful.

    The Right Way to Set Up Test Environments: Test Automation on Dynamic Environments

    The test automation world needs a game changer. You can make your car faster, but if the road is full of potholes, you won’t be able to get that fast or far. Test automation is similarly limited by the pace of environment creation, with its infrastructure, application, data and all.

    This challenge becomes critical when adopting DevOps and testing stops being a stage in the pipeline, instead becoming an overall pipeline service. QA is no longer about being the gate between development to operations. It’s about overseeing the quality of the product during its entire life cycle. This means automated tests should always be able to run, and the environment they run on must be dynamic.

    The solution to this challenge is environments as a service (EaaS). EaaS is all about packaging the environment, including the application, infrastructure and data, as an entity that can be provided as a service to a business process. Once defined, it can be provided on-demand for manual or automated users. Since it’s a well-defined entity, it also can be decommissioned when the process finishes using it. Playing an important role in making infrastructure horizontal, EaaS is also the key for paving the road for brilliant test automation.

    If you think you should get more from your test automation initiative, it may be a good time to make EaaS part of your automation strategy.

    — Maya Ber Lerner

  • Can DevOps and MSPs Work Together?

    Can DevOps and MSPs Work Together?

    DevOps and the adoption of MSP-based IT operations are both surging trends in the IT industry today. They are also trends that contradict each other in some key respects. Here’s a look at what MSPs mean for DevOps, and vice versa.

    DevOps and MSPs: A Primer

    MSPs, or managed service providers, are companies that provide IT services on a contract basis. They’re a solution for organizations that want to outsource some or all of their IT needs. In turn, organizations can avoid maintaining IT staff in-house or reduce the size of their IT staff.

    DevOps, of course, is an approach to software delivery that emphasizes collaboration between developers and IT operations.

    MSP-based IT solutions and DevOps are both on the rise. The MSP market is growing by about 12.5% annually, while DevOps is growing at a rate of around 18.6%.

    Cross-Purposes?

    The markets surrounding MSPs and DevOps may both be growing. But that’s not because they feed each other. On the contrary—in one key respect, MSPs and DevOps are at odds.

    How? It’s simple. The more your company outsources IT operations to MSPs, the less it can embrace DevOps. That’s because, when IT operations are outsourced, companies lose control over the culture and process surrounding IT. Customers don’t typically dictate to their MSPs which culture, tools or processes to use. As a result, it’s not possible for companies with a heavy reliance on MSPs to “do DevOps.”

    To put it another way: You can’t do DevOps if you have no IT Ops department because you outsource IT operations to an MSP. Likewise, if you maintain some IT staff but rely on MSPs to provide IT services that your employees can’t deliver, you can only embrace DevOps insofar as your in-house IT staff can support it.

    DevOps-Friendly MSPs

    The fact that outsourcing IT operations to MSPs means surrendering control over your IT culture doesn’t mean that you can’t use MSPs and embrace DevOps at the same time. There are plenty of DevOps-friendly MSPs out there. If you’re into DevOps as well as the MSP model, you simply need to find an MSP company that works within the DevOps culture.

    Granted, DevOps is more complicated when some or all of your IT operations are run by a third-party provider. It’s more difficult to achieve the level of seamless communication and continuous feedback that DevOps demands when not all of your IT employees are in-house.

    But it’s not impossible, especially if you find an MSP who is committed to transparent communication, the sharing of data and collaboration in IT planning—as opposed to one that delivers managed IT services inside a sort of “black box,” without involving the client organization in any of its operations.

    Providing DevOps-friendly managed IT services requires more work on the part of MSPs. It also demands more initiative from their clients. But it can be done. And it’s an approach that more and more companies will need to embrace if the influence of both continues to rise within the IT industry.

    — Chris Tozzi

  • Value Stream Optimization: Putting Business in the Driver’s Seat

    Value Stream Optimization: Putting Business in the Driver’s Seat

    How value stream can be a major driver of DevOps success in today’s software-only world

    Even as DevOps reaches a pinnacle in acceptance (as of 2018, only three percent of firms had not adopted DevOps and had no plans to do so), many organizations are still struggling to get high-quality software out the door. DevOps has been promoted by many experts and consultants as a solution to the cumbersome inefficiencies that were common with waterfall methods. However, as is common for many operational activities, applying a new approach or technology doesn’t automatically resolve entrenched problems.

    Nowhere is this more obvious than in value stream management (VSM). This approach, which focuses on removing the waste from processes to increase their efficiency, sounds like a panacea for resolving software delivery challenges. In practice, it often falls short of its goals.

    Why is this? There is nothing wrong with the concept of the value stream. The idea of breaking down a complex process into its components and optimizing each one of them, thereby optimizing the chain, is a long-recognized pillar of lean production. In my view, the problem isn’t with VSM or even DevOps.

    The problem is that in moving from waterfall to DevOps and/or agile, many organizations figuratively “throw the baby out with the bathwater.” They are uncertain how to adjust for the extreme acceleration that occurs when teams move from rudimentary software processes to modern ones, so they either jettison important elements of software engineering or adapt them incompletely.

    Why Continuity Matters

    By way of example, consider measurements and metrics. The traceability and proactive remediation enabled by measurements and metrics are two of the most important aspects of software development. However, it’s not just the development and test metrics that matter, but also the business metrics—metrics of quality, user acceptance, security and ease of use. Since the dawn of software development, those two groups of data have been inextricably linked, but some modern software teams appear to have forgotten that lesson.

    No amount of process improvement metrics can ensure a software release is secure or that users will like it. Yet, if a major software upgrade is flubbed, it can impact operations so negatively that sales momentum stalls. (If the company sells the software it develops, the ripple effect is even stronger.) Even more disconcerting, after a major data breach, a firm can find itself out of business.

    Don’t get me wrong. I am not suggesting that flow and other DevOps metrics don’t matter, because they do—very much. I am suggesting that many teams and their development managers have become too divorced from the business metrics, which can have staggering negative consequences.

    From There to Here

    In the days of waterfall, development artifacts, defects and other data were warehoused in an offsite location, and after each release (which might take days, weeks or even months), teams would engage in several days of data extraction. Then, a separate team would attempt to map the development artifacts to the defects and enable traceability. After hours or days of organization and cross-mapping in Excel spreadsheets, reports would be generated for management and the C-suite. Once stakeholders and management completed their evaluation, the team would be cleared to make adjustments before the next release cycle.

    Now, our development paradigm has changed so dramatically that many teams aren’t giving metrics their due. That’s an issue with defect metrics that could impact the user experience or software reliability. It’s a serious, potentially game-ending problem when the metrics are related to security.

    Completing the Value Stream

    To create and administer the complete value stream, software teams must be supported in three areas beyond the traditional DevOps framework where iterative development activities are the primary driver of traceability measurements. With these elements in place, management can map their software teams’ efforts to the business metrics that are the final measurement of success:

    • Organizations must have a mechanism for integrating development and testing tools so that defect measurements can flow back to the development team in real-time—matching the speed of DevOps.
    • Critical software activities that still take place outside of development and testing processes, such as generation and approval of change orders, must also be monitored for inertia, with approvers being held accountable for holding up paperwork that delays releases.
    • Security defects must be given parity with, and preferably priority over, all other code defects. All defects that impact the user side, such as functionality and launch speed, are important business metrics. But security must be paramount.

    If you are a development team member reading this, or even a development manager, you may think business metrics and the related activities that happen outside your purview, don’t apply to you. I urge you to think again. Software has become a critical pillar of many companies’ operational success, and that connection is only growing stronger by the day. Everyone involved in software development, from the developer to the VP of Operations and on to the CEO, must view the entire process through the lens of business.

    — Joe Schulz