We’re excited to share our findings on Firebase Remote Config A/B Testing Experiments! If you’re having decision fatigue trying to decide which ad variant is giving better Retention Rate or LTV, read on as we discover the goodness of this feature. 

For starters, Firebase Remote Config A/B Testing Experiments is a feature that gives you a way to optimize your new gaming app’s feature or user experience for a business goal. Through that, Retention Rate and Events count will be the 2 main metrics tracked for performance comparison.

‍

‍

‍This constant A/B testing supports product team in making data-driven decisions to optimise Variant A or Variant B by adjusting the creative assets or features to better serve your business goals. 

An example business goal could be to track if  Retention Rate on D7 (RR D7) for Variant A and B and if higher RR D7 equates to a higher LTV. With this in mind, some of the research questions that you might ask are:

1. If RR D7 of Variant A is 10% higher than that of Variant B, does it mean that Variant A has a better group of users? 
2. Does this mean that LTV D1 of Variant A will be higher than LTV D1 of Variant B? 
3. If Variant A’s LTV D1 is higher than Variant B’s, can you confirm that Variant A is performing better? 
4. How can I compare the 2 variants with more metrics like Duration, LTV at any day, IPDAU, RPDAU, Pay Rate? 

‍

Of course, you are going to be asking 101 questions before making a strategic decision for the product enhancements. The above is just a tip of the iceberg. To do that, more data points are needed to support your hypothesis. Connect your analytics, attribution and ad network data sources to Cost Center for a bird’s eye view of all the metrics.

‍

Zero guess work. It’s time to make real data-driven decision.

‍

Further comparison can be done on 2 variant volumes from the same campaign source. You can now easily identify the better performing Variant at a glance. 

‍

‍

Since this feature release earlier this year, Firebase A/B testing became the must have tool for our customers. Testing experiments could be on the difficulty of various game levels, simple aesthetics testings on background colours. In less than a month, reiterations can be made based on the test results shortening the whole validation process to prepare for a new round of A/B testing again. 

With the right decisions made, our clients have improved their product which results in stronger adoption and retention. If you feel that you are still doing the guess-work every time when making decisions with Firebase AB Testing results, contact us now and our team will guide you to success! 

‍