DZone
Thanks for visiting DZone today,
Edit Profile
  • Manage Email Subscriptions
  • How to Post to DZone
  • Article Submission Guidelines
Sign Out View Profile
  • Post an Article
  • Manage My Drafts
Over 2 million developers have joined DZone.
Log In / Join
Refcards Trend Reports Events Over 2 million developers have joined DZone. Join Today! Thanks for visiting DZone today,
Edit Profile Manage Email Subscriptions Moderation Admin Console How to Post to DZone Article Submission Guidelines
View Profile
Sign Out
Refcards
Trend Reports
Events
Zones
Culture and Methodologies Agile Career Development Methodologies Team Management
Data Engineering AI/ML Big Data Data Databases IoT
Software Design and Architecture Cloud Architecture Containers Integration Microservices Performance Security
Coding Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks
Culture and Methodologies
Agile Career Development Methodologies Team Management
Data Engineering
AI/ML Big Data Data Databases IoT
Software Design and Architecture
Cloud Architecture Containers Integration Microservices Performance Security
Coding
Frameworks Java JavaScript Languages Tools
Testing, Deployment, and Maintenance
Deployment DevOps and CI/CD Maintenance Monitoring and Observability Testing, Tools, and Frameworks

A New Way to Beta Test

Product teams are having to make betas shorter, forego them all together, or slow down their release cadence to gather adequate customer feedback. Now what?

Justin Baker user avatar by
Justin Baker
·
Dec. 28, 16 · Opinion
Like (11)
Save
Tweet
Share
12.11K Views

Join the DZone community and get the full member experience.

Join For Free

it’s best practice for products to have some sort of beta, a way to collect customer feedback and test performance before releasing to everyone. in an era of continuous delivery, we are delivering new features and experiences more frequently and with less time to gather thorough customer and performance feedback. with this increased cadence, product teams are having to make betas shorter, forego them all together, or slow down their release cadence to gather adequate customer feedback.

challenges of traditional betas

  • coordinating opt-ins. it sometimes takes weeks or months to gather customer opt-ins to test new betas. you also have to organize the distribution of beta keys (i.e., for early access to games) and reminder emails.
  • organizing focus groups. getting feedback from focus groups is often time-consuming and expensive, creating a long feedback loop that lengthens the release process.
  • opt-out. if customers opt in to a beta and don’t like the experience, then they will want a simple mechanism to switch back to the production version.
  • granular betas. it is very difficult to do targeted betas based on user attributes or to perform incremental percentage rollouts of new beta features.

feature toggles

to overcome these challenges, smart product teams are beginning to run betas with feature flags and toggles. these are mechanisms for granularly controlling software releases, allowing you to control the timing and visibility of a beta release.

currently, many betas are tied to code releases and are managed by a config file or database.  this approach requires engineering time or custom mechanisms to opt-in users.

with feature toggles, you can empower product, marketing, sales, and even customers to opt-in new to a new beta experience. feature toggle beta test launchdarkly

in this simple example, you can use a toggle to control the visibility of a new beta feature. ideally, this toggle would be part of a user interface that could be controlled by a non-technical team member. the code itself could be deployed off and then turned on via the toggle. beta test percentage rollout with feature toggle launchdarkly

moreover, you can also use the toggle to control the percentage of users who get the beta experience. for instance, you could release the new beta experience and have it rolled out to 0% of users. you could gradually increase the rollout percentage from 1% to 5% to 20% and more, collecting customer and performance feedback along the way.

surfacing this beta control functionality in a user interface is critical for giving non-technical team members access to release controls.

regional betas

for a recent example of a targeted rollout, we can look at how pokémon go released their product country by country: first to the united states and then abroad.

this is a great use case for feature toggles because you can create targeting rules to determine which users receive the feature first. for example, i could create a toggle that is governed by the rule: “if users live in san francisco, then serve the new nearby pokémon feature.” this allows you to maintain different regional feature sets without having to deploy different versions of the application. it also allowed pokémon go to refine their algorithms and assess customer feedback before rolling out the new feature to a wider audience.

benefits of beta testing with feature toggles

  • empowered non-technical users. allow the sales, marketing, product, design, and business teams to turn features on for specific users, collect feedback, and control the business logic. this also substantially cuts down on engineering time.
  • production feedback for your beta tests. test features in production with limited user segments to collect customer and performance feedback.
  • incremental percentage rollouts. gradually roll out features to incrementally test performance and mitigate risk. if the feature is bad, toggle it off.
  • real-time opt-in and opt-out. allow users to opt in and out of beta tests in real time, controlled via the feature toggle. skylight provides a nice article on this.

getting started with toggles

conceptually, a feature toggle is relatively simple. you create a conditional in your code that controls the visibility of a code snippet. there are many open-source libraries that will allow you to get started. however, these libraries become cumbersome when you try to feature toggle at scale or restrict access to particular toggles. depending on your needs, you could consider a feature toggle management platform to provide a system for access control and mitigating technical debt.

BETA (programming language) Testing

Published at DZone with permission of Justin Baker, DZone MVB. See the original article here.

Opinions expressed by DZone contributors are their own.

Popular on DZone

  • Using JSON Web Encryption (JWE)
  • The 31 Flavors of Data Lineage and Why Vanilla Doesn’t Cut It
  • Kotlin Is More Fun Than Java And This Is a Big Deal
  • How To Avoid “Schema Drift”

Comments

Partner Resources

X

ABOUT US

  • About DZone
  • Send feedback
  • Careers
  • Sitemap

ADVERTISE

  • Advertise with DZone

CONTRIBUTE ON DZONE

  • Article Submission Guidelines
  • Become a Contributor
  • Visit the Writers' Zone

LEGAL

  • Terms of Service
  • Privacy Policy

CONTACT US

  • 600 Park Offices Drive
  • Suite 300
  • Durham, NC 27709
  • support@dzone.com
  • +1 (919) 678-0300

Let's be friends: