Usage Model Update, 03 August 2018

[fa icon="calendar"] Jul 24, 2018 6:45:31 PM / by Priori Data Science Team

Date live in Priori Data: August 3rd, 2018

Model(s) updated: Retention, DAUs, MAUs, ARPDAU

Platform(s) affected: iOS, Google Play

Here at Priori Data, we are committed to continuously improving the performance of our estimates in light of new data or changing market dynamics. On top of that, our methodology evolves as we grow and learn how to better model this type of data.

 

What did the update do? 

We’ve made changes to our model methodology for our usage estimates in both iOS and Google Play. Some of the major changes are listed here:

    • Quality of input data: We updated our training sets with new partner data and improved our preprocessing algorithms which filter out erroneous observations.
    • Model features: We’ve incorporated more sophisticated features such as app description, as well as made improvements in how we capture differences between verticals (i.e. categories) and seasonality.
    • Model assumptions: We’ve made changes with the way we model retention behavior past day 30 to represent differences in the retention curves for iOS and Google Play.
    • Model formula: We’ve improved the way we model DAU and stickiness in order to address previous overestimation in active usage.
    • Improved model for top performers: We modified our methodology to better model the usage and retention behavior of apps at the top of the market.
    • Bug fixes.

 

Impact on estimates

These changes have resulted in meaningful improvements in the precision and stability of our estimates. This has resulted in a downward revision of our DAU and MAU estimates for both Google Play and iOS for the majority of apps. However, for apps at the very top of the market, our estimates have tended to increase on average. As for retention, we now see a slight decrease on average day 1, day 7, and day 30 retention for both GP and iOS.

Additionally, we’ve seen the following improvements:

  • Less erratic trend behavior over time for apps
  • Less overestimation on iOS and Google Play for DAU / MAU

For accuracy, we generally see improvements across the board in terms of MAPPD (median absolute percentage point difference) for day 1, day 7, and day 30 retention for both platforms. A summary of the most accurate verticals and countries for day 7 retention are listed below (negative values in the last column indicate an improvement in MAPPD):

 

Top 5 Verticals (Categories) in terms of Google Play Day 7 Retention Accuracy

Mean Absolute Percentage Point Difference, Q1 2018

Top 5 Verticals in terms of Google Play Day 7 Retention Accuracy

 

Top 5 Verticals in terms of iOS Day 7 Retention Accuracy

Mean Absolute Percentage Point Difference, Q1 2018 

Top 5 Verticals in terms of iOS Day 7 Retention Accuracy

 

Top 5 Countries in terms of Google Play Day 7 Retention Accuracy

Mean Absolute Percentage Point Difference, Q1 2018 

Top 5 Countries in terms of Google Play Day 7 Retention Accuracy

 

Top 5 Countries in terms of iOS Day 7 Retention Accuracy

Mean Absolute Percentage Point Difference, Q1 2018

Top 5 Countries in terms of iOS Day 7 Retention Accuracy

 

Overall, better models means better accuracy. It's important to note that the changes in our estimates might affect some apps, publishers, and categories more dramatically than others. If you have questions about how a specific subset of our platform has been impacted, please contact us (info@prioridata.com).

Topics: Release Notes

Related posts