DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

Speed Up Mobile Testing Cycles With Test Sharding

What is test sharding? Learn the answer and see how it will increase the speed and coverage of your mobile app tests for continuous delivery.

Lingkai Shao user avatar by
Lingkai Shao
·
Nov. 08, 17 · Tutorial
Like (5)
Save
Tweet
Share
4.55K Views

Join the DZone community and get the full member experience.

Join For Free

It’s obvious that companies nowadays are investing more in automation solutions to streamline the workflow of build, test, deploy, and monitor mobile apps. As developing high-quality apps requires lots of intensive testing, companies will most likely highlight speed as one of the cornerstones when evaluating test automation tools and services.

With speed, you can either get the same amount of tests done in less time or cover more test cases within the same amount of time. It depends on your use case which one you want to achieve. When you parallelize your tests across hundreds of real devices, more devices and tests are covered in one single test run. But how to verify a lot of tests, say 100 or even 1000 tests, in less time? Test sharding is the answer.

Why Do Test Sharding?

Imagine that you have 1000 tests taking 100 minutes in total to complete on a single device. Wouldn’t this take too much time, keeping every team in the pipeline waiting for testing feedback? Now, what if we divide these 1000 tests to get them run on 10 devices (100 tests for each device)? It will theoretically take 10 minutes rather than 100 minutes to finish the test session and get the test reports.

In the end, the purpose of doing test sharding is to spread tests across devices and ease the testing workload so that you can execute automated tests faster, increase throughput, and scale continuous delivery with a short feedback loop.

How to Shard Tests on Bitbar Cloud

Bitbar Cloud comes with the ultimate scalability so that allows you to run tests across any number of real devices concurrently. As to test sharding, we have the out-of-the-box support for it. You can try it out with a free trial. 

With our Key/Value pair argument capability available from the Advanced Options when setting up a test run, you can use 2 new Key/Value pairs, numShards and shardIndex. And we will address handling these within the testdroid.properties -config file (available for editing by system administrator).

Key/Value pair for test shardingThe test runner supports splitting a single test suite into multiple shards, so you can easily run tests belonging to the same shard together as a group, under the same instrumentation instance. Each shard is identified by an index number. When running tests, use the -e numShards option to specify the number of separate shards to create and the -e shardIndex option to specify which shard to run.

For example, to split the test suite into 10 shards and run only the tests grouped in the second shard, you would give the following value to key/value pairs in test options. This can be done either in our Cloud UI or through API.

numShards=10
shardIndex=2

P.S. If you are not in the scope of doing test sharding but would love to run a specific test case rather than the entire test case, we have the option as well. Simple define which test package or class you want to execute in the Advanced Option.

run certain test package or test class

Device Availability

To get shorter testing cycles, there are two pillars. As you’ve seen, one is splitting tests and the other one comes to the devices. More specifically, device availability.

By the time you split tests across a few devices, you want to make sure every device is available so that all the shards can be executed simultaneously. If the tests are executed at a different time, then you are basically doing it sequentially and your so-called ‘test sharding’ will bring minimal benefits to you.

Bitbar Cloud hosts a public cloud that offers the highest number of unique device models with different SW and HW combinations. Beyond that, we are well prepared for the increasing demand of a single device model by providing several copies in our shared testing environment.

That being said, setting up a private infrastructure with a few dedicated devices is the best solution to maximize the device availability in terms of doing split testing. With access to this kind of private testing environment, you can distribute your tests to all of the connected devices to ensure the fastest possible test execution time and allow you to forward feedback to developers instantly.

To Sum Up

The nature of automation is to reduce the manual work that needs to be repetitively done and decrease lead time with expected outcomes. By using advanced techniques like test sharding, you can attain a more tightly closed feedback loop among DevOps teams.

Testing

Published at DZone with permission of Lingkai Shao. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Differences Between Site Reliability Engineer vs. Software Engineer vs. Cloud Engineer vs. DevOps Engineer
  • OpenID Connect Flows
  • Understanding gRPC Concepts, Use Cases, and Best Practices
  • The Role of Data Governance in Data Strategy: Part II

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: