devopsy.kinsta.cloud

Tag: mobile app testing

Investing in a Mobile Device Testing Lab

It will likely come as no surprise that app downloads are on the rise, with no signs of slowing down. Today’s consumers use apps for everything from ordering food, to shopping for clothes, to managing money and more.

In fact, according to a recent report by App Annie, consumers downloaded a total of 194 billion apps in 2018, up from 178 billion apps in 2017.

Yet, the same report shows that app retention rates after 90 days are low–ranging from 32% to a meager 4%.

Reasons for app deletion vary, but far too many apps, in a burgeoning sea of apps, are either hastily and incompletely tested or aren’t even tested at all. This can lead to poor consumer experience, deleted apps and even negative feelings about a brand itself.

Therefore, it’s critical for mobile app developers and practitioners to button up their testing practices before release. Otherwise, you run the risk that all the work you put into your app will only lead to the dreaded and overcrowded app graveyard.

Overcoming Today’s App Testing Limitations

The primary challenge prohibiting companies from properly testing their apps is access to real-time devices in the same instantaneous, maintainable and scalable infrastructure. The expectations around continuous testing in desktop, web and packaged applications is difficult to achieve in mobile testing without a device cloud, which delivers seamless access to real devices in an on-demand infrastructure and brings order to the chaos of managing multiple mobile devices in a test lab.

A cloud-based mobile testing lab provides a comprehensive solution that solves issues around mobile app testing, including too many device types and operating systems to test, as well as a lack of people and time to test. It offers real-time, continuous access to real devices for all mobile testing needs, resulting in higher quality apps and increased customer satisfaction and retention.

An added bonus is a cloud-based mobile testing lab frees app testers from initiating and monitoring automated tests. Basically, testers can automate their automation. This, in turn, allows them to perform other significant tasks that can increase user acceptance, engagement and experience.

Failure Looms with Little or No Testing

That being said, some organizations, in a rush to go to market, release an app with little to no mobile testing at all. They believe testing is too difficult, too time-intensive or too costly to be worth their while. Some think minimal web testing will keep them out of trouble when it comes to mobile. And although we all like to think that we write bug-free code, unfortunately this isn’t always the case.

Furthermore, there are those who will test their app and then release it before fixing all the bugs, thinking they can test and improve along the way. However, that way of thinking can be detrimental to a company and its brand. Localytics found technical failure as simple as an image not loading can lead to a 29% loss of customers.

Mobile App Testing Helps Ensure a Thumbs Up

The average U.S. adult, according to an eMarketer study, spends 3.75 hours on mobile devices each day, with nearly three of those hours using apps. Further, Statista estimates app downloads to grow to 258.2 billion in 2022. Clearly, the app market is thriving, but so is the competition. You’ll have a better chance of standing out in the crowd if your app works–flawlessly.

A mobile device testing lab could be one of the key solutions that will help separate the proverbial app wheat from the chaff. By testing apps in real time, you understand which types of devices are most affected should a crash occur. Analytics show what app features are used most, allowing you to prioritize what should be fixed first during the testing process.

The future of apps is in mobile, which is why mobile testing will continue converging with legacy applications such as web and desktop. Forward-looking companies need an approach that allows top-of-line technologies to work together as if provided by a single solution provider.

A mobile device testing lab helps you break through the barriers of app development. In doing so, your app has a better chance of doing what you say it will do–without crashes–hopefully earning your app a coveted spot on users’ home screens instead of the trash.

— Dan McFall

November 18, 2019
Rookie Mistake: Not Managing Mobile Customer Feedback
Managing mobile customer feedback effectively could be the single-most important success factor for your company and customers. Call it addiction or habit, but every passing day we encounter new data confirming the growth of mobile app usage and total hours spent interacting with them. In the “2015 U.S. Mobile App Report,” the CoreMetrics team has shared some useful data metrics regarding digital media consumption growth and app usage. A couple of notable themes or trends are:
1. Americans across all age groups spend increasing time on Mobile Apps: By mid-2015, Americans on average had spent approximately 68 hours per month on their mobile devices. That number rises even further to more than 90 hours per month among millenials.
2. A whopping 50 percent of time is spent within a single app and about 90 percent on the top five apps: Not only is the average time spent on mobile app usage increasing, it is concentrated in a few limited apps. A staggering 50 percent of their time is spent working within a single app, and that number tapers off rapidly as popularity, utility and quality decreases.
It is evident from this data that you want customers to include your app on their home screen. That means not only having a compelling utility, but also ensuring that the app is of the highest quality. In this article I will illustrate how you can accomplish that by incorporating mobile customer feedback loops in every stage of the development cycle and involving and engaging customers through your path to production. Along the way, we will look at various open source and commercial tools that enhance the ability to collect feedback at various stages of the continuous delivery pipeline.

Typical Continuous Delivery Pipeline

Most agile or lean software organizations have an automated continuous integration and delivery pipeline that helps manage the flow of release artifacts to production. While there might be nuanced variations across teams and enterprises, the typical path to production for apps or services is represented through the steps shown:

It will be useful to review this pipeline as we explore options to incorporate feedback throughout the path to production.

Continuous Delivery and Mobile Customer Feedback

Not too long ago, most negative user feedback consisted solely of poor ratings, comments and rants on social media platforms. Without having context, metrics and data, most development teams were left guessing on how best to reproduce and determine extent of the issue. This usually resulted in issues and features getting prioritized in an ad hoc manner.

Today, numerous open source and commercial tools are available that help solve the feedback problem. Most of them can be categorized in the following broad areas:
- Instrumentation
  Despite recent reports that Facebook made its Android App crash to test user loyalty, most app crashes are not of the deliberate nature. There are a plethora of crash testing and reporting solutions, the most popular being Crashlytics (part of Twitter Fabric), Apteligent (formerly Crittercism), TestFlight (Apple) and HockeyApp (Microsoft). They each provide a client-side SDK to capture and report crash reports and a server-side service to to manage and analyze those reports.
- Phased Releases – Alpha, Beta, Staged Rollout
  Android application developers can take advantage of the alpha, beta and staged rollout capabilities available as part of Google Play. For alpha and beta tests, production APK is not required. Developers have a number of options to target specific groups for their alpha and beta tests. They also can utilize feedback channels available within the app to obtain feedback. Staged rollout allows a developer to progress from alpha and beta testing to production in a phased manner, constantly collecting feedback and metrics, comparing against baseline (if available) and progressing to 100 percent in production based on satisfactory results. Developers on the iOS platform can avail of beta testing capabilities provided by TestFlight and can invite up to 2,000 beta testers via email to test the App pre-production.
- Direct In-App Feedback
  Gone are the days when email, messenger or app store comments provided the only semblance of feedback. Customers increasingly expect to communicate via the app, and provide feedback without leaving it. Meanwhile, mobile app developers have better to tools to scale that feedback and manage it effectively to address bugs, prioritize features and get context—screen shots, system information, etc.—for specific feedback. HelpStack is an open source framework that makes in-app customer feedback relatively easy, so they can report issues, attach screen shots, embed device information and more. Customer feedback can be reported via email, or help desk integrations are available.
  Apptentive, Crittercism, HelpShift, UserVoice and Parature are some of the notable commercial solutions that enable in-app feedback.
- Metrics & Performance
  When business analytics do not provide provide adequate answers, diving deeper into technical data might provide insights into what is going wrong. Two key mobile customer feedback metrics worth tracking is app uptime and performance. App uptime helps identify customers facing crashes while performance issues could be impacting engagement rates. Some of the tools mentioned in this article do provide an APM solution that enables monitoring of these key metrics.
- Experimentation & Analytics
  A/B testing or multivariate testing is a data-driven experimentation approach to promoting app feature changes. While some of the above tools enabled you as a developer to better listen to your app community, A/B split testing allows you to validate your hypothesis and invest further in features and capabilities that demonstrate lift.
  Apptimize, Optimizely and MixPanel are few of the available commercial products that offer multivariate testing capabilities that can be incorporated in some situations without code modification or app updates. They also provide abilities to compare test campaigns against baseline and promote features that demonstrate positive feedback.
Incorporating Mobile Customer Feedback In Continuous Delivery

Leveraging data-driven tools throughout the continuous delivery pipeline increases the odds of a quality and high-utility mobile app. Even in situations where it is not possible to expose apps prematurely, it is possible to “dog food” your app—a term referencing a team or company that actively utilizes its own product with the intent of testing it. For example, frequent builds tied to developer commits enable alpha build testers and other team members to download and test the app. Meanwhile, beta builds can be downloaded by other members of the company. As release candidates emerge the beta community can be expanded to include members of the app community.

So take steps to incorporate mobile customer feedback loops at every step of the development cycle and garner a majority of the app user’s engagement time. Would love to hear how you and your teams are leveraging tools and techniques to build better apps and continuously listen to customer feedback.
April 8, 2016
Testing Mobile Apps? Consider a Mobile Device Cloud

Market demand for mobile app development services is expected to grow at least five times faster than internal IT organizations’ capacity to deliver by 2017, according to Gartner. As a result, enterprises must use the most efficient, effective and secure methods possible for their mobile app development processes, including mobile app testing.

Today’s users expect the highest quality user experience from their mobile applications, so testing should be a top priority for enterprise IT departments that want to keep up with these expectations. Mobile testing is more complex than legacy application testing, however, because of the myriad platforms, networks, OSes and devices on which the applications run. To ensure their mobile apps perform reliably for every user, businesses must test on key variations of this growing universe of mobile devices.

Sharing a single large pool of mobile devices across large or geographically distributed testing teams creates a number of challenges that risk bleeding IT and QA testing budgets dry. Mobile device access management has been plagued with inefficiency and risk, requiring IT teams to purchase an unreasonable number of devices to cover their enterprise’s growing needs. Inevitably, many of these devices remain unused or become lost, which can stall testing and result in idle mobile engineers.

In addition to the cost and inefficiencies of having multiple mobile devices, security has become a top concern for today’s enterprises, thanks to the rise in data breaches. To avoid complications, enterprises must hold mobile testing practices to the highest security standards. A data breach during app testing can create significant cost for an enterprise and harm its reputation. Plus, for industries that must adhere to Sarbanes-Oxley, HIPAA or other federal privacy regulations, a data breach or unauthorized user access can create a regulatory compliance issue.

The Mobile Device Cloud Option

A mobile device cloud may be the answer to the cost, security and compliance issues enterprises face in their mobile app testing environment. It can serve developers looking to test and debug applications and QA teams performing manual or automated testing, and facilitates collaboration among these groups and other stakeholders such as marketing and customer support.

A mobile device cloud offers organizations four significant benefits: security, accessibility, speed and cost. Let’s explore each in a bit more detail:

Security

On-premises, private mobile device clouds access a centrally managed pool of devices and mobile apps and enable testing within an enterprise’s corporate infrastructure and behind the firewall. This creates a highly secure environment for proprietary apps and sensitive data throughout the testing process, which is managed in-house. In addition, private, cloud-based mobile testing can improve compliance with regulatory standards by reducing the chances of a data breach or unauthorized user access.

Accessibility

Private mobile device clouds can still meet the enterprise need for global access—developers, testers and mobile engineers can securely access centrally managed mobile devices from locations anywhere across the globe without compromising corporate data security.

Speed

Technology advancements in mobile application testing platforms now allow automated, continuous testing simultaneously on multiple devices. This allows private mobile device clouds to be even more effective in increasing an enterprise’s app testing speed and quality without compromising security.

Cost

With IT budgets for QA and testing increasing 9 percent over just the last year, reaching 35 percent of enterprise IT team’s total budget in 2015 according to Capgemini’s “World Quality Report 2015,” IT teams need affordable options that still maintain efficiency, effectiveness and security. Private mobile device clouds often are much more cost-effective than off-premises, public device clouds, while also providing an agile, efficient and effective solution.

BI Intelligence predicts that mobile phone sales will surpass 2 billion in 2016, which means demand for enterprise applications will continue to increase rapidly. More enterprises already have begun using on-premises, private mobile device clouds and are seeing the benefits of this secure, agile, affordable option for their mobile application QA needs.

For those enterprises that haven’t yet considered private mobile device clouds, 2016 is a great opportunity to upgrade your mobile testing with an improved solution and to meet both employees’ and customers’ high expectations for enterprise mobile user experience.

March 3, 2016