iOS A/B testing

What is iOS A/B testing

iOS A/B testing is the process of running a controlled experiment comparing one or more variations of an iOS app against the original, with the goal of improving a specific metric, such as taps, engagement or in-app purchases. The experiment is delivered to a selected percentage of the application's install base. App users are randomly distributed between the original and variation experiences without knowing they are part of a test. Once a statistically significant difference in user behavior emerges, the results of each variation are compared to determine which performed best against the predetermined goal.

This method can be used to test changes across any aspect of an application where a measurable goal can be improved, including user interface (UI), onboarding flow, content and messaging, and many more. For example, testing changes to an app's first time user experience, or onboarding flow, would help determine what factors improve conversion rates through each step of the onboarding process and will lead to increased retention over time.

iOS A/B testing can also be used to experiment with new features and releases to measure their impact before submitting them for App Store approval. This phased rollout process is made possible through Code Blocks, which encapsulate a new experience into a conditional code statement that allows one to deliver that experience to a controlled percentage of an app's install base.

Benefits of iOS A/B testing

A/B testing allows iOS developers, marketers and product managers to gather data about their install base and use that data to serve up higher converting, optimized experiences. This process of experience optimization helps mobile teams achieve key business goals, such as engagement, retention and monetization.

Mobile app A/B testing makes it easy for testers to demonstrate return on investment (ROI), because tests are created with a quantifiable goal in mind and results are easy to track and measure. Teams become more efficient, as testing ensures developers and designers focus their time building experiences that are proven to convert.

What elements can be tested?

All categories of iOS applications can benefit from A/B testing, and depending on the app, teams may prioritize testing different elements. Within the app, nearly every element can be altered and tested. Product managers, marketers and app developers often experiment with:

  • Visual elements: Pictures, videos and colors
  • Text: Headlines, calls to action (CTAs) and descriptions
  • Layout: Arrangement and size of buttons, navigation and forms
  • Onboarding: How a first-time user engages with your app
  • User flow: How a user gets from point A to B within your app
  • New features and releases: The latest version or update from your app

Best practices for iOS A/B testing

While there is no tried and true formula for building the perfect app experience, best practices can be used as a starting point when considering what to test. Some iOS A/B testing best practices include:

  • Eliminate distractions: Mobile users want to get from point A to point B quickly. Use limited screen real estate wisely to keep them focused and engaged on the task at hand. Fewer elements create less distraction from the conversion goal.
  • Focus on the CTA: CTA text and placement resonate differently depending on the audience
  • Show, don't tell: Too much text can create friction for app users. Instead of text, focus on immersing users in an interactive app experience that leads to deeper engagement.
  • Aim for the global maximum: Test with the overarching goal of the iOS application in mind, not the goals of individual screens or elements.
  • Provide consistent experiences: Make testing changes consistent throughout the user flow to improve conversions at every step of the process. If you have a web experience, make sure users see a consistent experience across devices.

iOS A/B testing with Optimizely

Optimizely is a platform that makes mobile A/B testing easy, allowing users to build, create, measure and connect experiences across devices. Test it out on your iOS application today.