iStock-868434080-2

Usage Model Update 4.0, May 13th 2019

May, 8 2019

Subscribe and stay up to date

Date Live in Priori Data: May 13th 2019 (for platform users), May 20th 2019 (for API users) 

Model(s) updated: Retention, DAU, MAU, ARPDAU

Platform(s) Affected: iOS

Here at Priori Data, we are dedicated to the continuous improvement of our estimates and evolvement of the methodology behind them. With the ever-changing dynamics of the mobile app market and the increasing amount of data that we collect on this, our models are becoming ever more complex. It is, for this reason, we are releasing the usage model updates for iOS and Google Play separately, in order for us to fully recognise the differences between the two platforms and adapt our models accordingly.

 

What changes did we make?

Input data:

  • We have included new data points into our training sets in areas in which we could not previously obtain usage actuals, particularly at the top end of the market.

Model features:

  • We have engineered more intricate features and introduced new algorithms which improve how we capture app behaviour and the accuracy of our models.
  • We have changed the way seasonality is defined in the retention model to increase granularity.

Model formula:

  • We are now modelling MAUs and retention days directly which avoids inter-model dependencies and captures the behaviour of these metrics through machine learning.

Top Apps:

  • Top apps are now estimated using a separate model and a dynamic method for determining the position of apps within the different models has been implemented; this ensures apps are incorporated in each model to the correct extent.

Infrastructural changes:

  • We rewrote modelling scaling infrastructure to allow for large batch data processing, enabling more granular data to be included in pre and post-processing, resulting in faster model iterations.

Bug Fixes.

 

Impact on Estimates

As retention estimates are percentage points we have used median absolute error (MAE) to determine the error rate in these estimates. However, as MAU figures represent individual users, we have used the relative absolute error (RAE) to determine the MAU error rate. This normalises the difference between apps with very high and very low user estimates.

These changes have made vast improvements in the accuracy and stability of our estimates across the board. Our overall RAE for MAUs has been decreased to 18.15% from 108.66% and our retention MAE has seen a decrease across all the days that we provide estimates for (1, 7 and 30). The results of our more direct modelling methods show that retention estimates have decreased overall but have increased for the top performing apps.  MAU estimates are showing a downward revision of for both iPhone and iPad across each category which combats the overestimation that was seen in the previous model. The tables below show a summary of the top performing and most improved verticals and countries for the month of January 2019.

 

Top 5 Verticals (Categories) in terms of iOS MAU Accuracy 

Vertical Relative Absolute Error
News 7.42%
Social Networking 12.81%
Sports 12.82%
Games/Casino 12.87%
Games/Card 13.36%


Most Improved 5 Verticals (Categories) in terms of iOS MAU Accuracy

Vertical Relative Absolute Error
Games/Racing 24.70%
Games/Sports 17.05%
Games/Music 42.85%
Games/Action 15.28%
Games/Family 24.48%

Top 5 Countries in terms of iOS MAU Accuracy

Country Relative Absolute Error
Kenya  1.46%
Uruguay 1.91.%
Sri Lanka 1.96%
Venezuela 2.05%
Ecuador 2.59%

Most Improved 5 Countries in terms of iOS MAU Accuracy

Country Relative Absolute Error
China 35.38%
France 20.63%
Italy 22.03%
Saudi Arabia 10.32%
United Kingdom 23.35%

Top 5 Verticals (Categories) in terms of iOS D1 Retention Accuracy

Vertical Median Absolute Error
Food & Drink 4.56%
Games/Adventure 4.74%
Games/Music 4.78%
Games/Racing 4.83%
Educational 4.89%

Most Improved 5 Verticals (Categories) in terms of iOS D1 Retention Accuracy

Vertical Median Absolute Error
Lifestyle 7.92%
Educational 4.89%
Food & Drink 4.56%
Travel 5.27%
Health 5.86%

Top 5 Countries in terms of iOS D1 Retention Accuracy

Country Median Absolute Error
Brazil 5.62%
United States 5.71%
China 5.72%
Italy 5.92%
Peru 5.97%

Most Improved 5 Countries in terms of iOS D1 Retention Accuracy

Country Median Absolute Error
Kenya 11.09%
Venezuela 13.59%
Uruguay 7.33%
South Africa 8.62%
Thailand 6.35%

Overall, more specialised and improved models result in higher accuracy. It is important to be aware that the changes that we have made can affect some apps, publishers and categories more than others. If you have any questions about how anything in particular has been impacted by this update then please reach out to us at info@prioridata.com.

View all posts

Subscribe and stay up to date

Speak Your Mind

Subscribe and stay up to date