Many IT organizations currently utilize licensed automation tools, but providing licenses to all QA and development personnel is impractical. Typically, licenses are allocated to designated automation engineers who create and run tests as needed. This creates a challenge in allowing any team member to execute tests as required. With most automation engineers located offshore, additional licenses would be needed for onshore test execution, which can be expensive. Furthermore, ensuring high code quality, adequate coverage, absence of code smells and thorough security scans is crucial. Configuring a CI pipeline to invoke automation tests can help reduce license costs. CI tools sit in front of automation test tools, eliminating the need for individual licenses. In this article, I demonstrate this using GitHub Actions (GHA) as the CI tool. GHA is open source but incurs costs based on runner usage. I propose an innovative solution to eliminate runner costs too. Additionally, integrating code quality tools like SonarQube and CodeQL into the pipeline ensures security scans and highlights issues before deployment.
The major advantages of implementing a CI pipeline are:
No Automation Tool User Access is Required for Invoking Tests
Automation tools that require licenses can only be used by licensed individuals. However, when integrated with a CI pipeline, any user with CI access can initiate jobs without needing a direct license. This approach optimizes the management of the license pool, reduces costs and extends the tool’s utility for test automation. For instance, all developers with GHA access can now invoke the automation tool without requiring individual licenses.
Eliminating the Requirement for QA Resources to Constantly Oversee UI Test Execution on Their Desktops
During UI test execution, some automation tools prevent testers/developers from navigating away from the UI screen, blocking them from performing other tasks on their computers. By utilizing the GHA pipeline to invoke the automation tool, this restriction is removed, allowing developers and testers to work on other tasks while the automation runs in the background. After execution, an email with a detailed, user-friendly PDF test results report is sent.
Before the CI Pipeline came into the picture, the manual test execution setup had effectively reduced our hours from XXXX to XX, but we still needed to allocate XX hours because a QA resource is needed to monitor the screen during test execution.
However, an automated CI/CD integration reduces these XX hours to zero, as the CI pipeline manages sessions, eliminating the need for manual monitoring during test execution.
Other Benefits of Using GHA as a CI/CD Tool for Automation Tests
Access to GHA Free With User GitHub Account
GHA is a GitHub feature, which is enabled by default and hence no explicit access is required for GHA — it supports CI/CD workflows directly into the repositories.
Cost Savings: Eliminating AWS Infrastructure Cost
For a CI/CD pipeline, we must have a runner (virtual machine for running CI jobs) integrated, which is typically provisioned in the cloud, such as AWS incurring additional cost that grows as the load increases. However, with this solution, we are utilizing the organization’s Win365 machines as the runner and eliminating additional infrastructure runner cost incurred while using cloud services. All automation engineers are provided with Win365 machines for installing and executing automation scripts.
Replacing AWS Runners with Win365 runners
Automation team members for installing automation tools if provided with a virtual machine, which mirrors your desktop and incurs fixed charges. QA resources use this cloud instance only during business hours, leaving it free for the remaining 16 hours daily. By leveraging VMs as CI pipeline runners during this free time, we can significantly reduce AWS usage, eliminating the need to spin up new EC2 instances.
The below diagram depicts cost savings — here we are incurring a fixed VM monthly cost of $55, hence the pipeline using these VMs will incur zero runner cost. But if a new AWS instance were being used as a runner, we could have incurred an $80 monthly cost.
Cost Savings Calculation
Firewall Constraints and Automation tool Configurations
With GitHub being managed internally within the organizational network, we can avoid any security system issues as well as automation tool API permission issues, which is usually a challenge with externally hosted CI/CD tools.
Code Quality Checks Implementation
In the CI pipeline, we can implement CodeQL and SonarQube code quality tools avoiding any separate need for testing. CodeQL is a heavy tool and takes time for the pipeline to be executed. So, it is always recommended to trigger CodeQL in a separate pipeline while the results can be viewed within GitHub itself.
What is CodeQL
Post running the pipeline, errors if any highlighted by CodeQL can be viewed.