devopsy.kinsta.cloud

Tag: software testing

  • How AI is Transforming Testing Today

    How AI is Transforming Testing Today

    Currently, there is much debate about how artificial intelligence (AI) will transform our lives. One of the technology’s key benefits lies in its power to transform traditional approaches to app development and software testing. Gartner identified AI-driven development as one of its top three strategic technology trends for 2019.

    Digital transformation is dependent on better-quality software, and software testing matters more than ever. However, the growing complexity of our software-driven world means that new, smarter and quicker forms of testing are critical in the race to build better software faster. But in a hyper-connected and complex environment, this is no simple task, and this is where AI-driven testing can help.


    AI Transforming Testing

    Like any new technology, there are exaggerated claims. However, when it comes to testing, AI is already delivering real improvements and making intelligent automation a reality. AI has moved far beyond an academic concept and is driving business benefits as opposed to hints to those organizations that have adopted it. Below are four of the key ways that it’s already transforming testing.

    Auto-Generation of Test Scripts

    There are undoubtedly complex test cases for which AI can’t auto-generate all the code. However, AI can easily auto-generate the code that clicks the buttons on the screen, fills out a form, logs into the app and other similar functions. Organizations that have deployed AI-driven testing are able to auto-generate 80% of code reliably, delivering substantial efficiency improvements. Along with auto-generation, AI achieves auto-maintenance: Tests are rebuilt when something in the digital product changes, ensuring continuous quality is maintained and further reducing the resource burden on human testers.

    Test Optimization

    If you’ve ever used Amazon, then you know that recommendation engine technologies are not that futuristic. AI can be applied to testing in the same way to help companies determine which test is most likely to find a defect based on the risk information gathered. Rather than taking a haphazard approach to testing, AI enables you to home in on the areas at risk—not only delivering efficiency gains but also ensuring a quality experience that delights.

    Release Impact

    Neural networks, combined with test history and data from current test runs, can predict how an upcoming release will impact users. For example, will customer satisfaction go up or down? Armed with this information, companies can make any adjustments necessary to ensure their users continue to be delighted by their digital experience.

    Customer Impact

    This is a key benefit of deploying AI-driven testing as you can determine the impact on the business from release updates. Neural networks, machine learning and other AI techniques can be used to understand how technical factors impact business performance. For example, are load times negatively impacting conversion rates? By understanding how releases will affect the business, you can make changes that will quickly have a positive impact.

    What Does the Future Hold for AI-Powered Testing?

    In addition to the core benefits that AI-driven testing is already delivering, organizations can expect to see additional benefits within the next 12 months that will further accelerate their ability to fully automate the entire testing life cycle.

    Failure Determination

    Current testing compares the result of a test with a pre-defined known good result. That methodology simply will not work with dynamic systems, such as an internet of things (IoT) device that interacts with multiple other devices which are constantly changing. AI is a key part of the solution: A well-trained AI system can determine whether a real user would consider the actual result correct or not.

    Persona Testing

    One size doesn’t fit all, and the ability to understand how your app reacts under different personas will allow organizations to understand what parts of the app are particularly good or bad for different personas. Armed with this information, you can optimize your app for your target audiences, ensuring an experience that delights.

    As every organization races toward digital transformation, delivering a high-quality digital experience is essential for survival. To create compelling digital experiences, companies need tools that can navigate applications intelligently, learn where quality issues are most likely to occur and identify the critical data correlations that will help developers resolve issues quickly. This is where AI is already delivering significant advantages to organizations that have adopted it.

    It’s clear that AI-driven testing is not a passing fad. Rather than taking a wait-and-see approach, it’s time to jump in and experience how AI can help transform app development and software. With AI-driven testing making a real and measurable impact on the business, you can’t afford to be late to this revolution.

    — Antony Edwards

  • OverOps Employs Machine Learning to Measure Application Reliability

    OverOps Employs Machine Learning to Measure Application Reliability

    OverOps is adding a series of dashboards to its analytics platform that employ machine learning algorithms to identify which software application anomalies in a pre-production environment are likely to have the most impact on a production environment.

    Company CTO Tal Weiss said one of the biggest challenges DevOps teams face is prioritizing which issues to fix first in an application. The Reliability Scoring capability being added to the company’s namesake platform for analyzing code employ machine learning algorithms to keep track of introduced errors, regressions and slowdowns during every phase of pre-production. In addition to enabling DevOps teams to visually identify the root cause of an issue via a single click, that data is then used to identify which issues will impact reliability.

    Weiss noted that the Reliability Scoring can also be employed to set thresholds of acceptable application performance that would also be used to determine whether there is an issue. The data collected using that capability then can be fed back into a continuous integration/continuous deployment (CI/CD) platform to set up gates that would then prevent suspect code from being promoted, he said.

    The primary benefit, however, should be a major reduction in the number of troubleshooting instances any DevOps team should need to engage in after an application is deployed in production, said Weiss. In fact, the Reliability Dashboards will provide a central mechanism through which developers, testing professionals and IT operations teams can collaboratively track and measure the progress of an application development project, he added.

    The Reliability Scoring tool OverOps developed employs “micro-agents” that operate between application code and the hardware. Weiss said those micro-agents can not only capture data that was never previously available, they deduplicate, classify and gate critical issues as they move into staging and production. The Reliability Dashboards visualize that data in Grafana, a set of open source tools for visualizing data. DevOps teams can also opt to feed that data into any existing dashboards they may already have in place.

    In general, Weiss said the metrics surfaced by the Reliability Dashboard are critical to justifying investments in DevOps over the long term. Organizations expend a lot of time and effort making the transition to DevOps. But most of them don’t really know to what degree they are getting better at building and deploying applications. Reliability Scoring dashboards provide the historical context required to make those assessments, he said.

    Far too many DevOps teams are still starved for data needed to make critical decisions. In the absence of that data, teams address issues as they arise, rather than based on the impact they might have on the business. Worse yet, it’s difficult to identify new issues over those that might be another manifestation of a known issue that suddenly becomes a much higher priority.

    Regardless of the root cause of any problem, however, DevOps teams would do well to remember things that get measured are usually the first things to get fixed.

    — Mike Vizard

  • How Can DevOps Enable Your Software Testing efforts?

    How Can DevOps Enable Your Software Testing efforts?

    Changing market dynamics are pushing the need to include DevOps in your software testing exercise to help meet business objectives in providing a consistent and flawless user experience.

    Digital transformation is not an easy task for business enterprises. There are myriad issues to conquer, such as department silos, legacy systems and a well-entrenched work culture that often refuses to align with the business goals. Needless to say, the imperatives of reaching out to the market quickly and with quality products often are not achieved because of these issues. To eliminate the bottlenecks that end up blighting business enterprises, a growing number of organizations are adopting DevOps, which involves an end-to-end automation of processes, be it for development, testing or deployment.

    DevOps takes the Agile methodology a step further by involving all the stakeholders of an enterprise. The DevOps testing strategy encompasses continuous improvement of software products to address the changing customer preferences.

    In DevOps software testing, the aim is to detect and eliminate glitches proactively to reduce the cost and time to fix issues that can lead to a poor user experience. In the highly competitive digital market, a poor user experience can stymie a business’s market readiness.

    However, businesses should realize that the DevOps software testing approach is a journey and should be leveraged to enhance the user experience on a continuous basis.

    What’s the Key Value Expected from DevOps?

    Considering that the development and operations teams of an enterprise each have different work cultures, systems, goals, tools and approaches, the quality and delivery schedule of applications can be thrown off. Hence, the significance of DevOps testing strategy comes into play to bring synergy between the teams and to enhance the quality and delivery of applications consistently.

    DevOps calls for the integration of development and operations teams, wherein QA is made everyone’s responsibility. The synergy helps in breaking the departmental silos and posits the role of QA as an enabler of value addition and to achieve customer delight. This not only ensures quality, but also enables faster turnaround on application development process. Ultimately, enterprises are better equipped for the changing market dynamics and challenges with their business applications.

    Key Drivers to Adopt DevOps

    George Spafford, research director at Gartner, said:

    “A DevOps initiative must focus on business requirements and not on ‘doing DevOps for the sake of DevOps,’ wherein the methods and tools become more important than what customers need. Organizations must avoid the all-too-common mistake of launching a DevOps initiative before establishing that a business reason exists to do so. For example, instead of focusing on release rates and doing things faster, start with the business value by asking what that will enable.”

    Hence, it’s important to work with a DevOps strategy that resonates with the business objectives. These are some key drivers for considering DevOps.

    • Quick deployment of quality applications across devices, platforms, operating systems and networks.
    • Increasing competition necessitating the faster release of apps.
    • Emerging technological platforms whetting the appetite of customers for better products.
    • The emergence of complex applications with footprints in the physical and virtual world.

    How Does DevOps Enable Effective Quality Assurance?

    • It makes QA a collective responsibility rather than being assigned to a single department or organization.
    • It enables better collaboration between teams through a shift of culture.
    • It ensures faster identification and elimination of glitches.
    • It is responsible for accelerating the time to market and keeps a tab on the software quality through customer feedback.
    • It provides continuous improvement of software through timely releases.
    • It reduces cost due to the absence of rework.
    • It optimizes the utilization of resources and prevents waste.
    • It helps to cut down the software development life cycle (SDLC) by adopting quick sprints.
    • It ensures continuous development, testing, integration, deployment, monitoring and innovation of applications.

    The Need to Measure DevOps

    Digital transformation has necessitated the need for rapid innovation, development, testing and deployment of applications. Post deployment, appropriate customer feedback should be considered and absorbed into the product to improve its performance further. The success of DevOps QA depends on the level of automation, not to speak of the degree of collaboration among various stakeholders.

    In fact, enterprises should use a robust automation software to execute quality assurance across platforms and devices. The automation software should be able to test the code as part of the development sprint on a variety of devices and test environments. The DevOps QA approach should be measurable based on certain metrics to ensure its effectiveness. The measurement platform can include the following:

    • The QA team dealing with continuous integration should be able to work with all the tools to find out glitches.
    • The entire setup should be simple to facilitate quicker learning, adoption and execution.
    • The test data including dependencies should be easy to access.
    • The entire build, deploy, release, verify and rework processes should be streamlined.

    Conclusion

    In the fast-changing digital landscape, the objective of reaching out to the customers quickly and with better quality of products is paramount. This is where the DevOps specialists turn things around with continuous improvement, innovation and deployment.

    As Gartner’s Spafford rightly stated, “DevOps challenges conventional IT thinking with its lack of a standard definition and approach, its constant evolution, and its requirement for acceptance and management of risk.”

    — Komal Lopez

  • Continuous Testing at More than 200 mph

    Continuous Testing at More than 200 mph

    “The old principle of tinkering with an instinctively designed car has long since been superseded by systematic testing of every major component and structure – both before and after the car is fully built and ready to race.”

    —“Understanding F1 Racing,” Formula1.com

    While speed is the most obvious shared goal between racing and software development teams, you might be surprised at the striking similarities between how these teams achieve that goal. Each relies on top talent, funding and innovation to reach the speeds at which their competition cannot, but there’s another oft-ignored component that’s quickly becoming one of the greatest differentiators between first place and everyone else: testing.

    The way F1 teams test their cars has gone through a transformation just as monumental as what we’ve seen in the software world over the last decade. The ability to continually test every major component and structure of a race car or piece of software is crucial, but the ability to quickly respond to the data that comes out of those tests is just as important. It’s continuous testing before, during and after the rubber meets the road—or the software hits the marketplace or app store—that determines the winners.

    The Evolution of Testing

    In the aforementioned Formula1.com article, an all-too-familiar scenario to those in the software world is described:

    “By midway through the first decade of the 21st century, a typical Formula 1 testing programme had become a major exercise in both manpower and logistics…spiralling costs, which invariably resulted in development work that was invisible and meaningless to fans …”

    Replace “Formula 1” with “software” and the above excerpt is just as factual for those who don’t know the struggles of yesterday’s approaches to software quality, and just as painful for those who personally experienced them. Manual processes, insufficient test automation technologies, shared unrealistic environments, lack of trust between teams and out-of-control costs are constraints of the past for today’s top-performing teams, but they remain for many still relying on legacy testing approaches and technologies.

    What Continuous Testing Looks Like Today

    Some of the racing enthusiasts from Tricentis’ leadership team were invited earlier this year to meet the Haas F1 racing team during their 2018 preseason testing at Circuit de Barcelona-Catalunya. We witnessed firsthand the remarkable transformation that Haas and F1 racing as a whole have embraced. Track testing is where, as Formula1.com says, “the steady evolution that happens to all Formula One cars during the course of their life begins, a constant improvement of tiny details and setup.” This constant evolution and iterative, rapid innovation is exactly the goal of high-performing software organizations today.

    After spending the day at the track, we noted the following approaches and methodologies that we see in innovative continuous testing and DevOps practices today:

    • F1 cars are continuously delivering data and feedback to the organization.
    • The cars have millions of data points and thousands of sensors, providing actionable data.
    • These data points and sensors are grouped into core processes, and isolated by key questions.
    • Test runs are done with the cars in as realistic environments as possible.
    • The speed at which that data can be received, interpreted—and most importantly—acted upon, is the difference between high-performing and low-performing teams.
    • The tiniest changes to the car make enormous differences. Nobody is showing up in an entirely different car, they’re making minute changes.
    • These cars cannot fail on race day or the race is lost.

    Today’s high-performing organizations are treating the delivery of software in the exact same manner as the world’s leading racing teams. Innovative technologies are being put into each stage of the delivery life cycle that provide invaluable feedback that helps enterprises better understand risks to their business. Being able to rapidly analyze—and act on—this amount of data has become imperative for avoiding disruption. Testing software in environments that are as realistic and as close to production as possible is mandatory, and those environments need to be available when they’re needed most. These needs have driven the adoption of what’s known as continuous testing—an evolution of traditional test automation that has proven unable to support software’s increased complexity, and expectations around the pace of development and delivery.

    What Questions Are F1 Racing and Software Enterprises Asking?

    While each have their own unique challenges, there are many similar questions that need answers—in real-time, and at any development, testing, or release stage—that the racing and software industries share.

    • How fast are we today and how fast will we be tomorrow?
    • How fast is our competition, and how fast will they become?
    • What are we measuring today that’s impacting the business?
    • How quickly can we introduce change?
    • What data are we using to validate that change?
    • What’s the risk of not changing?
    • How low can we get our mean-time-to-repair?

    The fastest time to market—or time to the finish line—is how you win the race. Your developers or engineers don’t just build the fastest car, or the most popular app to today’s standards and then call it a day. They’re constantly looking at what speed, UX, functionality or accessibility will be needed to be the best tomorrow, and then iteratively adding those improvements as quickly as possible. They don’t stop engineering, ever. And testing shouldn’t be performed any differently. Each new feature, condition, region, device, persona, currency, language and OS that your app needs to offer, integrate with, accept or run on is going to need continuous testing before, during and after that app’s release.

    Whether you’re responsible for the quality and performance of a billion-dollar race car or the applications for a billion-dollar enterprise, speed and quality are your path to the winner’s circle. Continuous testing ensures that you remain there.

    — Noel Wurst

  • The Era of Intelligent Testing

    The Era of Intelligent Testing

    The state of the art in software testing has not kept pace with advances in software development over the past 10 years. Modern quality assurance (QA) groups struggle to play their role in a world of continuous delivery, leading to more defects in production, low QA morale and excessive testing overhead. Thankfully, innovations in machine intelligence and test automation have laid the groundwork for a better model—one that will allow QA to keep pace with development, increase productivity, improve product quality, and raise customer satisfaction across the software industry.

    Why We Need a New QA Toolset

    Existing QA solutions were built for a world where software changed infrequently and functionality was highly specified and documented.  This is no longer the case.

    Products Are Evolving Faster Than Ever

    Several forces are converging to accelerate development. Software as a service (SaaS) has removed many of the issues around operating system compatibility, packaging, deployment and versioning that consumed significant development time during the installed software era. Consider that in 2016 Amazon Web Services delivered more than 1,000 new features and services—a pace that would be impossible with packaged software.

    “Development cycles are getting shorter and new features are coming faster than ever…There simply isn’t enough time to test.”

                                                 – QA Leader at Fortune 250 company

    The proliferation of open source and cloud-based features as a service has also reduced the time that developers spend building generic functionality. At mabl, for example, the thought of building our own authentication engine never crossed our minds; we use Auth0. Instead of building our own data processing pipeline, we use Apache Beam and Google Cloud Dataflow. Rather than building our own ML framework, we use Tensorflow. And the list continues; reusable components and cloud-based services save us years of engineering effort, allowing our developers to deliver product features rapidly.

    Continued adoption of agile methodologies has further increased the efficiency of development teams. A recent McKinsey study of 1500 software projects demonstrated that agile teams are 27 percent more productive than those that follow methodologies such as waterfall.

    Finally, continuous integration and continuous delivery are accelerating the pace of change in software. A recent survey by Atlassian suggests that more than 50 percent of software teams are already using continuous integration and/or delivery, and more than 32 percent have plans to move to CI/CD. The same survey indicates that more than 50 percent of organizations are able to push changes to production daily.

    In the end, improved developer productivity means that software products are evolving faster than ever. Unfortunately, this places increased pressure on the QA function. As one QA leader from a Fortune 250 company said in a recent interview with mabl, “Development cycles are getting shorter and new features are coming faster than ever. And we have five productive developers for every QA engineer. There simply isn’t enough time to test.”

    Existing Automation Frameworks Impose Too Much Overhead

    QA rightfully relies on automation frameworks to improve productivity; tools such as Selenium, Appium and JUnit now enjoy widespread adoption. More than 80 percent of the respondents to a recent mabl survey use Selenium today, and QA managers are under constant pressure to increase automation. Unfortunately, while Selenium and other frameworks have helped many companies increase their QA velocity as compared to manual testing, these frameworks have significant flaws:

    • They all require specialized scripting expertise, and given the lack of available talent, teams have very limited capacity for QA automation.
    • They provide limited context in test results, which leaves teams with very little information to help them triage and address issues.
    • At scale, they require their own infrastructure, which takes time to provision, operate and scale.
    • They are tightly coupled to attributes of the product under test that change frequently (xPath, CSS, etc.), resulting in constant maintenance and false positives.

    These flaws only become more painful as the pace of development and change accelerates, compelling many teams to limit their use or discard them altogether, resulting in lower product quality.

    mabl survey – satisfaction with existing testing tools and processes (n = 104)

    “Please indicate the extent to which you agree with the following statements”

    Time Pressures Lead to Unhealthy Team Dynamics

    People in testing roles are increasingly at odds with their peers in development. Unable to keep up with the continuous pace of development, QA becomes a bottleneck to product velocity, leading to tension with their peers in development. This constant pressure also starves QA of its ability to make the very investments in automation and process improvements that are needed to improve throughput. The feeling of always being behind with little hope for improvement often leads to low morale within the team.

    Gaining consensus on the standard of quality has also become difficult for teams, since agile teams often steer clear of detailed functional specifications for each release, preferring instead to focus on user goals and requirements (see: Agile Manifesto). Because user goals and requirements are open to more interpretation, QA and development can disagree on not only the severity of defects but also the validity of the tests themselves.

    Modern software teams are also more open to evolving requirements during a given release cycle (see: “Responding to change” in the Agile Manifesto). Evolving requirements should drive updated test cases, but this requires deep engagement and coordination for already stretched personnel. Rather than testing for quality versus specified behavior, we rely much more on ad hoc and manually intensive exploratory testing to look for defects.

    There is Hope for QA

    While faster feature development, shorter release cycles, evolving requirements and unclear accountability for end-to-end quality all present real challenges for QA, a new generation of QA tools are emerging to confront these challenges head on.

    It Will Be Easy to Create and Maintain Tests

    Unlike existing test automation frameworks, next-generation QA tools will not require specialized scripting expertise. Instead, they will offer intuitive interfaces that allow anyone to quickly create and manage tests. This will help teams expand their test coverage and evolve their tests quickly as their products evolve.

    Tests Will Adapt Seamlessly to Change

    Because they use much more sophisticated, durable methods of simulating user behavior in automated tests, next-generation QA tools will not suffer from the brittleness associated with existing automation frameworks. Tests will not be tightly bound to individual elements within the front-end codebase that change frequently. Rather, they will use machine intelligence to create and maintain sophisticated models of tests, enabling tests to adapt the tests rather than fail when xPaths and other locators change.

    QA Will Run in the Cloud

    Today, teams struggle with the performance, cost and operational overhead of on-premises testing systems. Next-generation QA tools will take advantage of on-demand cloud computing resources to execute tests faster and more efficiently. They will tap into powerful data processing and analytics services to analyze test results and deliver insights. They will be delivered completely as a service, offloading the management and operational burden to tool vendors.

    QA Will Be Part of the Delivery Pipeline

    Next-generation QA tools will be tightly integrated into automated CI/CD pipelines orchestrated by Jenkins, Spinnaker, CircleCI and CodeShip. Tests will trigger automatically whenever teams make changes to the product under test and on-demand. The tools will notify team members when there are potential issues so that they can address them before they affect users. It will be simple to run tests and compare results across builds and environments. Decisions around releases, deployments, promotions and rollbacks will be largely automated.

    The Definition of Quality Will Evolve

    Beyond validating specific assertions in tests, the next generation of QA tools will use machine intelligence to automatically detect and highlight potential regressions in applications. As a result, we’ll think less about tests “Passing” and “Failing” and more about the extent to which changes improve or degrade the user experience. We will focus less on code coverage and more on product coverage. We’ll look beyond basic counts of passing and failing tests in favor of a more on holistic approach to assess the risk associated with a given build or deployment.

    Tools Will Be Deeply Embedded Within Our Teams

    Next-generation QA tools will effectively serve as extensions to development teams. They will share test results and insights through Slack, Jira email and other common communications channels. They will take feedback and training from the entire team—including product, development, QA and customer success—and incorporate what they learn into their tests. Most importantly, next-generation tools will describe tests and results in plain language, and they will provide useful context to help developers reproduce and fix issues, just as an effective QA teammate would.

    The Era of Intelligent Testing

    Software quality assurance has never been easy, and the accelerating pace of software development in recent years has made it even more challenging. Thankfully, a new generation of tools are emerging to address this challenge. Incorporating cutting-edge machine intelligence, delivered from the cloud and embedded into the modern development workflow, these tools will dramatically improve the effectiveness of QA, ushering in a new era of efficiency and innovation across the software industry.

     

    mabl survey – user sentiment (n = 104.  For simplicity, combined “Agree” and “Strongly Agree” into “Agree” category,  combined “Disagree”, “Strongly Disagree” into “Disagree” category.)

    “How satisfied are you with the following aspects of the testing process at your company?”

    — Dan Belcher

  • 13 Best Practices of Successful Software Testing Projects

    13 Best Practices of Successful Software Testing Projects

    Thorough testing is vital for quality assurance, and best practices in software testing can yield high-end software at the end. The current post is giving fine clues regarding the prevailing best practices in software testing projects. 

    Recent cyberattacks have proved that security has paramount importance for any software life cycle. The best QA and testing practices can help software development companies to cut the worst incidents considerably.

    Here are 13 best practices to ensure successful software testing projects.

    Rely Only on a Controlled Security Environment for Testing

    Today, many software development companies and freelancers prefer cloud-based testing to capitalize on the cloud’s benefits. Unfortunately, the cloud has tons of vulnerabilities, particularly when the public cloud is in use.

    Software development agencies instead should invest on testing infrastructure and create a secure testing environment locally to avoid the internet and cloud-based vulnerabilities.

    For mobile app testing, regional testing labs and libraries are the best alternatives, where manual and automated testing take place securely and safely.

    Carry Tests Throughout the Software Development Cycle

    Testing only on the QA stage of the software designing process is an inadequate approach. Testing must carry with each major output/feature of the software. Plan the testing schedule right from the beginning stage of the development process.

    Early testing can catch bugs early and cut the defects early. It improves the quality of software and cuts the labor in the final stage of QA, and reduces the cost of quality maintenance. Moreover, it builds the confidence among the team of developers to include some innovations in the product.

    Break Tests in Small Fractions

    When you carry tests frequently and throughout the cycle, take smaller tests to save time and resources. Many tests are longer than we think, so carry such tests in smaller fractions such as UI testing, function testing, security testing, UX testing and so on.

    Write Tests for Maximum Coverage

    Write all types of tests to cover the maximum, if 100 percent is not possible. When you write test cases for valid conditions as well as unexpected conditions/behavior of apps under the test, you can cover maximum.

    Moreover, write test cases in the requirements step for analysis and designing step, too. This way, your requirements also become testable.

    Carry Regression Tests

    You must recognize and segregate your test cases for regression testing. A quick and effective manual regression testing is beneficial for the quality assurance of the. During regression testing, use a bug graph (number of bugs against time for different modules) to predict the most possible buggy part of the app.

    Programmers Should Not Write Tests

    Technically, tests must be written before coding. Programmers should not write tests or test cases for their own code to reduce  the chances of biased tests or a lack of creativity in the test. Tests should carry tests with own perspectives.

    Note and Report All Tests

    When you carry manual testing or automated testing, keep records of all tests. Note all test progress and observations in a file, and incorporate these observations into the final test reports. Thus, you can release absolutely unambiguous test reports and required details.

    Some of the things that should be noted include the bug’s symptoms, the effects it exerts on the entire system, and all possible solutions to make testing meaningful.

    Clean the Test Code Prior to Final Release of Software Product

    When an application under the test requires changes in core code to carry a test successfully, it must be cleaned once the testing process is completed. Therefore, record-keeping of testing code is a highly beneficial practice. In addition, save time and resources by cleaning the testing code from the core code at the time of the deployment of the final application.

    Isolate the Application from Developers

    During the development stage, designers and developers make constant changes mostly in a collaborative manner, particularly when the MVC paradigm has followed. Testing the application within the development environment is not advisable. For unbiased and complete testing, you must isolate the application from the design and programming environment for a declared period of testing.

    Customize Testing

    Write custom testing solutions for each project according to needs and their possible use cases. A module in an app running on a smartphone has different use cases than the tablet environment, so write the test accordingly.

    Accumulated Skilled Testers

    Highly skilled testers in the team is a welcome move for the success. Your testing team must have functional and automated testers with the latest testing skills in their respective areas.

    Similarly, for performance testing, you should have mixed talents with skills on LoadRunner and JMeter testing. Moreover, you should look for additional skills in your testing team such as UFT, HP ALM, Selenium and other open-source tools for advanced testing.

    Build a Target Operating Model for Testing

    Define testing processes, methods, tools, RACI matrix and communications for your testing project. Acknowledge all developers and team members of the QA department regarding these, their project goals and the responsibilities allocated to them.

    The process is termed as the target operating model for testing, and it helps in the successful entry of any new team member to work toward the common objectives.

    Have a Mix of In-house and Offshore Teams

    Collecting a testing team with required skills and investing in testing infrastructure is not always possible with all software development companies, particularly in developed countries where wages are higher.

    Therefore, you have to rely on outsourcing testing/QA team and their infrastructure. However, not all kinds of testing are possible with offshore teams, and you have to mix the tests and talents with offshore and onshore for successful testing.

    For instance, the behavior of the consumers/users of some products are different between the west and the east, and you need to carry such test cases in the west with the help of local or in-house team through applications that were built by the offshore team.

    Conclusion

    We have explored the best practices to enhance the testing projects for better quality and security of your software in the modern age. Experienced and expert software testing developers are mandatory to hit your testing goals.

    About the Author / Tarang Vyas

    Tarang Vyas is a head of the development team at software development company Perception System. He has been in the industry for 15 years. Connect with him on LinkedIn.

  • Testing: Your Secret Weapon for DevOps Success

    Testing: Your Secret Weapon for DevOps Success

    I spend virtually all day, every day, speaking with IT leaders who recognize that their bottom line ultimately rests on their ability to accelerate the delivery of innovative software. That’s why they’ve invested so much in digital transformation initiatives—including hiring the best and brightest developers, scaling Agile, driving DevOps adoption, automating the continuous delivery pipeline, and all the other components involved in moving from ideation to delivery as rapidly as possible.

    In most cases, this investment has exposed opportunities to optimize culture, process, and tooling—but not yet delivered the level of acceleration and flexibility that IT leaders expect. I’m frequently asked: “Now what? I think we’re ‘doing’ DevOps. But what else is needed to deliver on faster innovation?”

    Something as unsexy as “transform testing” is not what people expect to hear. As someone who’s been in the software testing industry since 2003, I know that software testing is typically seen as a necessary evil, not a strategic advantage. However, I’ve also witnessed how it can be a secret weapon for surging ahead of the competition.  

    The dirty little secret in the IT industry is that despite all the advancements in software development, software testing remains dominated by yesterday’s application lifecycle management (ALM) tools and manual testing—and they simply don’t meet the needs of today’s accelerated development processes. Transforming testing for the modern era represents a tremendous opportunity. It’s a low-hanging fruit that can help you increase acceleration, reduce risk and cut costs.

    Boost Acceleration

    Agile and DevOps initiatives aim to accelerate the process of delivering working software to the end user. However, even organizations that have adopted Agile and DevOps report dismal test automation rates: less than 20 percent for end-to-end testing. This means that after the rest of your software delivery pipeline is automated and optimized, an outdated testing process eventually emerges as the bottleneck. By transforming testing, you can increase your throughput significantly. I’ve witnessed many companies increase time to market by 55 percent by modernizing their testing process.

    Reduce Risk

    Now that software is the primary interface to the business, a software failure is a business failure. Yet, with today’s compressed and continuous delivery cycles, it’s simply impossible to test everything before every release—even if testing is automated. By aligning testing with your top business risks, you can automatically assess the overall impact to the core user experience and instantly determine if the release candidate has an acceptable level of business risk. This is critical for stopping high-risk software from impacting your prospects, customers, and partners.

    Companies still relying on traditional quality processes and platforms are usually covering only about 10 percent of their top business risks with their current test suite. A goal of 80 percent is typically achievable with about three months of effort.

    Cut Testing Costs

    There are hard costs associated with trying to retrofit outdated testing processes and tools into modern delivery processes. Many organizations try to bridge the testing gap by throwing an abundance of manual testers at the problem, typically through a global system integrator. By the most recent estimates, this approach consumes approximately 35 percent of an average IT budget—a total of $30+ billion per year, globally.

    A way to help testers achieve high test automation rates allows you to reallocate a large portion of that testing spend toward creative tasks which advance competitive differentiation. Many enterprises that transitioned manual testers into “scriptless” (non-technical) test automation have slashed testing costs—and eliminated one of the most significant constraints to achieving faster time to market.

    Get the Analysts’ Perspective

    Analysts at Forrester (e.g., Diego Lo Giudice) and Gartner (e.g., Joachim Herschmann and Thomas Murphy) have spent years advocating that transforming testing is a business imperative. If you’d like to get their perspective, I encourage you to review some of their latest research:

     

    About the Author / Sandeep Johri

    Sandeep Johri, CEO of Tricentis, has more than 25 years of experience as a technology executive and serial entrepreneur—including roles at Hewlett-Packard, Appcelerator, Oblix (acquired by Oracle), Blue Lane Technologies (acquired by VMware), and numerous other startups in Silicon Valley and India. Johri earned an MBA from Stanford University, a Master’s degree in Industrial Engineering from Wayne State University, Detroit, and a Bachelor’s degree in Mechanical Engineering from Pune University, India. Connect with him on Twitter.

  • 7 Best Practices for Continuous Delivery Success

    7 Best Practices for Continuous Delivery Success

    Continuous delivery refers to the development of software in short and ongoing build-test-release cycles along the deployment pipeline. Repeated testing of processes and scripts before deployment to production means most errors are discovered early on and have minimal impact. Plus, finding and fixing such errors is much easier with fewer changes per release—software is released faster and more frequently with fewer deployment problems due to a heavy focus on visibility, instant feedback and incremental changes.

    To accomplish this, companies must have a straightforward and repeatable deployment process. Most reliable, of course, is an automated deployment pipeline that can feed into any environment.

    We have identified seven continuous delivery best practices that are supported by a set of complementary task-specific tools, in what’s known as the “DevOps toolchain,” to automate the process from end to end and minimize human error.

    Version-Control Everything

    The native code deployment process generally includes a built-in safety net, preventing out-of-process, locally generated artifacts from entering the production environment. Every change a developer makes must be documented in the source control repository or it is not included in the build process. If a developer copies an artifact generated locally to a test environment, the next deployment will override this out-of-process change.

    This insistence on having a single source of truth—and a reliable one—creates a stable foundation for all development processes. However, the processes are only as strong as their weakest link. That’s why the primary golden rule for effective continuous delivery is to version-control everything. What works well for code also will preserve the integrity of configuration, scripts, databases, website html and even documentation.

    Build Binaries Once

    Not every organization’s build process is the same or uses the same tools, but every build process in a single organization must be completely consistent. Whether the build is a single file deployed to an automated test environment or a complex build with several different possible deployment versions, each build version should happen exactly the same way and result in unique binary artifacts.

    This one-time-only compiling eliminates the risk of untracked differences due to various deployment environments, third-party libraries or different compilation contexts or configurations that will result in an unstable or unpredictable release. Save the compilation phase output (the binaries) to a binary repository, from which the deployment process can retrieve the relevant artifacts.

    Deploy the Same Way Every Time

    An inconsistent deployment process can become a source of configuration drift across environments, especially with the rapid releases common in a continuous delivery system. As a result, valuable time and effort are wasted in identifying and addressing difficulties arising from the application working in one environment and not another.

    For a reliable process in all environments, then, the same set of steps must be repeated from start to finish. This ensures the same results in lower environments, with more frequent deployments (such as integration, QA, etc.), as in higher environments (pre-production and production), with fewer deployments.

    Smoke Test

    Smoke testing deployments is a very rapid way to make sure that the most crucial functions of a program work and will pass basic diagnostics. This non-exhaustive software testing (of all elements, such as services, database, messaging bus, external services, etc.) does not provide the same fine-grain comprehensiveness as full test suites; however, it can be run frequently and quickly, often in a matter minutes (rather than hours or days). This allows for a much quicker turnaround on what can become very time-consuming and basic issues.

    The faster the feedback loop works (especially with automated test tools such as Telerik, QTP, TestComplete), the higher the final product’s quality will be and the quicker it will reach its release-ready state.

    Deploy into a Production-like Environment

    Each new deployment (using, for example, IBM uDeploy, CA Release Automation, XebiaLabs, Automic) should be made into an environment that mimics as closely as possible the actual final production environment. This includes infrastructure, operating system, databases, patches, network topology, firewalls and configuration. By validating software changes in this type of detailed pre-production environment, mismatches and last-minute surprises can be effectively eliminated and applications can be released safely to production at any time.

    Instant Propagation

    Continuous delivery by definition must be an ongoing process of building, testing and releasing. Every check-in to a source control repository should trigger compiling (if needed) and packaging by a build server. This, in turn, should automatically initiate certain predefined testing until the software can either be marked as releasable or returned to development.

    Continuous integration tools (e.g., Jenkins / CloudBees, Bamboo, TeamCity) ensure that cascade of actions along the delivery pipeline is consistent, automatic and instant. Thus, development teams merge developed code and get feedback from automated test tools many times a day, for a more efficient deployment and update process.

    Include the Database

    No list of continuous delivery best practices would be complete without advising that the database be managed using the same basic protocols that ensure secure and reliable source code, task, configuration, build and deployment management. However, many times this is neglected because the database is unlike other software components or compiled code that are easily copied from development to testing to production. Yet, the database is actually a repository of the most valued and irreplaceable asset—our data—and preserving it accurately is imperative to continuous delivery.

    Although the database poses several unique challenges, specialized database tools such as enforced database source control (for all environments), a database build automation tool and database release and verification processes can ensure a stable resource in your DevOps chain.

    Considering these seven continuous delivery best practices will allow you to create a deployment pipeline with increased productivity, faster time to market, reduced risk and increased quality.

    The key to ensuring a quick, robust and accurately repeatable continuous delivery is, as we have seen, automation, deployment consistency, comprehensive testing and database version control. With these fail-safes in place, pushing the “release to production” button can be done much more often and with much more confidence.

    About the Author / Yaniv Yehuda

    yaniv-e1404303491389Yaniv Yehuda is the co-founder and CTO of DBmaestro, an enterprise software development company focusing on database development and deployment technologies. Yaniv is a DevOps expert who spent the last couple of years raising awareness about the challenges surrounding database development and deployment, and how to support database continuous delivery. Connect with him on LinkedIn and Twitter.

     

  • TechWell Announces the Full Program for STAREAST 2016

    The Premier Event for Software Testing and QA Professionals to be Held in Orlando, Florida

    Jacksonville, Florida (PRWEB) February 03, 2016

    TechWell Corporation, an industry leader in software development conferences and training, has announced the full program for STAREAST, one of the longest-running, and most respected conferences on software testing and quality assurance, to be held May 1–6, 2016 at the Renaissance Orlando at SeaWorld® in Orlando, FL.

    The week-long conference will feature over 100 learning and networking opportunities and covers a wide variety of some of the most in-demand topics: Test Management, Test Techniques, Test Automation, Agile Testing, DevOps & Testing, Testing the Internet of Things, Mobile Testing, Testing Metrics, Cloud Testing, and Performance Testing to name a few.

    Attendees can build a customized week of learning with a variety of topics available through pre-conference training classes, full- and half-day tutorials, testing-oriented concurrent sessions, and keynote presentations.

    One of the most popular features of STAREAST remains the top-notch keynote presentations.

    This year’s keynote speakers include renowned testing professionals: Keith Klain, David Dang, Lee Copeland, Isabel Evans, and Dawn Haynes. “The keynotes will allow attendees to gain insight into the evolution of the software testing profession and lessons learned along the way,” said Allison Wade, President, TechWell Corp.

    Keynote presentations include compelling topics like Lessons Learned in (Selling) Software Testing, Open Source Test Automation: Riding the Second Wave, The Evolution of a Software Tester: From Novice to Practitioner, Telling Our Testing Stories, and Lightning Strikes the Keynotes.

    STAREAST will also feature a Women Who Test full-day event on Friday, May 6 during the conference. Women Who Test was introduced at last year’s STARWEST conference in Anaheim and received rave reviews by the participants. This program will cover testing topics and will support both women’s personal and career journeys. The Women Who Test event covers prominent topics for female test/qa professionals, including: reengineering your life, growing your personal brand, setting goals, discovering your career superpowers, and how to problem-solve as a female in the tech industry where the ratio of men to women is drastically outnumbered at 7:3.

    For more information on STAREAST and access to the full conference program, visithttps://stareast.techwell.com.

    ABOUT TECHWELL CORPORATION:

    TechWell—through its conferences, training, consulting, community websites, and online resources—helps people and organizations develop and deliver great software. For more information, visithttp://www.techwell.com.

    For the original version on PRWeb visit:http://www.prweb.com/releases/2016/01/prweb13186748.htm