Device Testing Services at Global App Testing

Validating that your product works as desired across devices is an essential step for any business looking to launch a software product. But what kinds of device testing are there? What are the common pitfalls and mistakes teams make when they think about device testing? Why test across devices? And why crowdtesting?

At Global App Testing, we work with hundreds of businesses and ensure that they have access to sufficient device breadth in their product review to build products which are robust. Below is a quick introduction from our in-house writers about some of the things they’ve struggled with. 

If you’re considering device testing with crowdtesting, please get in touch! We’d love to talk about your overall approach.

What is device testing?

In this article, “device testing” is about software companies driving a great experience across different devices. (This is not an article about the hardware testing cycle). That would often include:

• Testing the experience across manufacturers
• Testing multiple versions of a single manufacturer
• Testing the experience on tablets if applicable
• Testing by hardware feature and spec, e.g. scanning a document with a lower definition camera
• Testing with different integrated accessories relevant to your user journey such as VR headsets, wearables, headphones, home devices 

Most organizations will have a multi-focused device testing process which accounts for functional issues, user experience, accessibility, and other blockers such as connectivity.

Types of devices and things to include in test scope

Global App Testing quality assurance (QA) teams acknowledge and understand how diversity in terms of devices, operating systems, user behavior, languages, and geographies impacts quality. 

Device Categories for Device Testing

Let's look at the key device categories and what makes each one a unique testing challenge:

Mobile phones and tablets

With a large number of devices being manufactured every year, QA teams need to cover a large variety of models, form factors, languages, screen resolutions, and operating systems. 

Below is the list of critical test cases based on our experience with mobile application testing: 

1. Operating systems and versions: Consider various platforms and OS and their last three versions, such as Android and iOS testing. 
2. Form factors: Test your application across devices with different screen resolutions and aspect ratios, and with devices with foldable screens.
3. Input methods: Validate apps with input methods, including finger touch, stylus pen, face recognition, and fingerprint sensor.
4. Manufacturers:  Local testing should account for the fact that certain brands dominate in specific regions. For example, Xiaomi devices running MIUI are widely popular across South Asia, Southeast Asia, and parts of Europe, making them critical for QA teams targeting users in these markets. 

Wearable, headphones, and VR devices

Wearables, headphones, and VR are used in fitness tracking, immersive entertainment, gaming, and hands-free communication. In our experience, these devices have a unique set of challenges due to their user expectations for accessibility and interaction modes: 

1. Wearables (e.g., smartwatches): Test syncing reliability with other devices, user interface (UI), and the effectiveness of pairing and connection.
2. Headphones: Validate the connection process, volume and playback controls, audio quality, and how they interact with in-app audio features.
3. VR/AR gear: Test the user movements, loading times, and memory usage during extended use sessions.

IoT and real-world devices

IoT(Internet of Things) devices are everyday devices that can send, receive, and process data from the internet. They can connect to wifi or Bluetooth to transmit information. 

To simplify testing, GAT teams have broken down this category into 3 key areas:

1. Smart home devices: Test for stable Wi-Fi/Bluetooth pairing and discoverability via mobile apps.
2. Environment variables: Simulate signal loss, obstructions, and distance to assess real-world performance.
3. Cross-platform configurations: Ensure the device maintains state across layers, such as connection status, etc. 

So far, we have learned what devices we should target for testing. Now, let’s see how testing should be done and what factors to consider given the extensive range of devices.  

Getting the devices you need

One critical weakness of most testing processes is that teams tend to reuse a single QA device (and therefore don’t really undertake device testing). 

It can be simpler to manage devices by outsourcing this to the crowd. Across the hundreds of thousands of people in our network, you can cherrypick hundreds of devices without having to buy and maintain them.

Device Testing by Environment

Environmental factors must be taken into account during the testing phase, as users from different regions worldwide may have varying conditions, including location, network, and language. A few of those environmental factors are: 

1. Network conditions: Verify the behavior of the application under changing network conditions, such as 3G, 4G, etc.
2. Geographic variation: Validate that location-dependent features like time, currency, etc, work as expected.
3. Light/Dark Mode: Ensure that the user interface, including text and colors, is good in both dark and light modes.

Configuring Your Device Test Run

After you have a list of test devices, environments, and user attributes specific to your customer's needs, you then need to determine how the test cases will be run and who will run the tests. To do this, these are the basic steps:

1. Define the testing purpose: Determine the focus of testing, i.e., functionality, performance, usability, or regression.
2. Select test devices: List down the devices, OSs, models, and versions most used by your customers for testing. 
3. Choose between simulators and real devices: Separate out the list of test cases where you can use simulators instead of real devices.
4. Environment setup: Set up proper network conditions, OS, and user roles for your test execution.

For a single device run, consider the following test goals based on your needs:

1. Style-focused testing

Testing styles pre-defined test mixes that outline the scope and focus of testing activities. They can range from exploratory testing with no formal test case documentation to end-to-end testing with detailed test cases or even customer surveys to conduct product reviews. These testing styles help teams determine the right amount of effort and save time during the testing process.

2. Subject-focused testing

Before executing the device testing suite, it is critical to know exactly what you’re testing. This usually starts with figuring out the key parts of your app that matter most, i.e., core business flows like logging in, searching, or checking out.

Think about how your app looks and interacts through a variety of devices. This is called responsiveness testing, and it includes verifying that the user experience is smooth when the user switches to dark mode or uses the application in landscape mode, irrespective of screen size.

Additionally, include hardware features like cameras, sensors, and GPS in the test suite to ensure thorough coverage. 

3. Issue-focused testing

This approach includes focusing on high-priority bugs in core user flows. These are the bugs that can have the heaviest impact on business. A few examples of these defects include device-specific bugs, performance issues, compatibility problems, and connectivity issues.

What does crowdtesting offer for device testing?

In-house vs Crowdsourced QA teams

Some of the advantages of crowdtesting for devices are as follows:

1. Access to the devices you need without having to buy or manage them:  Avoid the hassle, cost, and liability of paying for hundreds of devices by using a provider with them on-file
2. A more representative device sample: Target the specific devices, operating systems, and configurations that your real users rely on
3. Integrated localization testing: Combine device coverage with language and regional checks to validate the holistic user experience.
4. Faster and thorough test coverage: thousands of distributed freelancers enable you to test on hundreds of devices in a short time.
5. Scalable and cost-effective: Outsourcing teams save the cost of recruitment, training, infrastructure, and long-term employee benefits.

Resourcing your device testing via crowdtesting

You might be wondering, “How exactly does crowdtesting plug into our existing QA process?” The good news is that it’s built to integrate, not replace. At Global App Testing, we’ve helped top software businesses like Facebook and Microsoft enhance their in-house testing capabilities by introducing a hybrid QA testing approach, which combines in-house testing with crowd testing. 

Let’s see how these two teams work together: 

Testing in-house

Your internal QA team remains the foundation for structured, repeatable testing, owning the test planning, test case design, and automation. When it comes to testing on scale in terms of people and devices, the in-house team faces challenges of hiring and budget. 

Device testing via other data products

Crowdtesting comes into play when broader device, location, and user variability is required. Combined with test coverage analytics, real usage data, and device performance insights, crowd-executed scenarios help prioritize high-impact devices and user journeys. 

In-house and crowd teams are combined to form a structured and scalable QA strategy.  

Get started today

Delivering high-quality products across a range of devices, platforms, and environments requires an experienced and large QA team. While internal QA teams bring structure and strategy, they lack the scale and diversity to do comprehensive device testing.

That’s where Global App Testing can make the difference:

• Fast and scalable testing on real devices across global locations
• Real-world scenarios and edge case validation
• Smooth integration with your existing QA process

With GAT as your testing partner, you can achieve test coverage across a range of devices and locations.