Askblu.ai explain data science and game difficulty tuning: a new paradigm

In this article, Dominique Busso, Founder & CEO of real-time personalization platform askblu.ai explains how a key process in game design -- difficulty tuning -- is being disrupted by data science and machine learning, and how game companies that embrace this new paradigm will get an unfair advantage in player retention and monetization.

Player retention is key to monetization of mobile games

In the mobile gaming world, and especially in the casual and hyper-casual games market, all economic models - freemium with IAP, ADs, subscription, even premium - rely on player retention. You know the magic formula: cost per install < lifetime value (for a player).

You need to lower your CPI and increase your LTV. The more players that stay in your game, the more opportunities there are for ads to be watched. Users that continue to use games are also more inclined to make in-app purchases, stay subscribed and ultimately increase your monthly revenue.

Let’s take a look at an example. You have a game with $1 CPI and $0.30 ARPDAU, and you pay for 50K new players per day for 30 days. If your retention rate is 35% on day 1, 15% on day 7 and 4% Day 28, your revenue minus the cost of UA will be $738 thousand (which means a $1.49 LTV).

If you can increase Day 7 by 2% and Day 28 by 1% (so you get 17% on day 7 and 5% on day 28), then your revenue minus cost of UA will be $952 thousand - that’s a $214 thousand gain in income (+38%) with LTV increased to $1.63.

Player retention remains the big challenge in mobile games

During the hyper-casual track at the last Pocket Gamers Connect in London (January 2020), several speakers explained how they have numerous tools and specialists to help them acquire players, but that retention continues to be a big challenge.

According to analytics tool for game developers GameAnalytics, good retention rates are considered to be 35% on day one (50% for hyper-casual games), 11% on day seven and 4% on day 28. From all of those difficult-to-acquire users, 90% will stop playing after only seven days.

Difficulty tuning is key to player retention

During the first and second game sessions, many factors will influence the likelihood of a player continuing to play your game. FTUE (First Time User Experience) is most significantly impacted by look-and-feel, artistic direction, UI & UX, IP, and whether the gameplay corresponds to what the player was expecting.

For players that continue after the first sessions, difficulty becomes a key factor in retention.

Most game designers know the “Cognitive Flow” concept - each game has its own difficulty progression and each player reacts to the difficulty based on their ability, how they feel and according to how much they are enjoying the game.

Players will churn (definitively stop playing the game and never come back) for two main reasons:

• frustration
• boredom

Challenge one: many studios are tuning game difficulty blind

Testing the difficulty of all the stages in a game can be very time-consuming. You predominantly rely on feedback from your team and some testers - but are they actually representative of your real players?

Setting up an analytical tool in your game with events before each stage and a stage funnel in the analytics web portal might show you that you are losing a high number of players after, for example, level 10, but it won't show you why. Was it too difficult, or too easy?

You might adjust the power of the opponent, put more moves in a Match3 stage and reduce the goal to make it easier for players, but how do you know that it really is?

Most importantly - how can you be sure that the difficulty is optimized for your overall player base at every stage in your game?

The solution: data-driven difficulty analysis

When you receive data about players starting a stage, including how they finish it (winning, losing, quitting), you can extract very important information about how your real players behave in-game and how they react to changes in the difficulty as you tune it. You can use statistical tools including Guassian regression to work on the tuning of all stages.

This data-driven analysis and decision-making can be done manually either by developers or by an analyst or data scientist - but it takes a lot of time that game studios don't always have.

An automated solution saves a lot of this precious time and can give you daily feedback so you can see if your game difficulty stays optimum as you change your player acquisition channels.

Challenge two: players are very diverse in casual games

We know that retention depends heavily on difficulty tuning - a frustrated player will churn (too difficult), a bored player will churn too (too easy). But players in casual and hyper-casual games are very diverse.

You can try to 'hard code' some 'rules' into your game, for instance, 'make the stage easier after three losses,' but nothing beats a data-driven approach. This will allow you to discover which players tend to churn after having lost only two levels, and which stay on regardless of the result. Some players, for example, tend to churn once they get an easier version of the level following a loss - they want to earn that victory!

The solution: real-time player personalization using machine learning and predictive analysis

The diversity of players’ behavior, asking for a personalized experience that cannot be handled, or at most imperfectly, by arbitrary rules, can be done by machine learning algorithms that learn from the behavior of your real players and make churn predictions.

Analyzing data and dynamically picking the best option for each player can be automated, using machine learning, statistics, automation processes and cloud computing.

Decision trees, logistic regression, clustering, neural networks and statistics are common tools that can be used for such automation, where the detection of criteria that maximize a figure of merit - like player retention - is performed automatically, without the need to dig through the data by yourself.

Conclusion: a paradigm shift for game difficulty tuning

Rather than tuning a game blind and trying to program “expert” rules, game developers will get significantly better results using a real-time data-driven approach.

If your game delivers the same stage difficulty to all players, you will lose players that would have potentially stayed had you been able to detect in real-time when they become frustrated or bored. With this data, you can set the difficulty of the coming stage to be either easier or harder according to the needs of each individual player.

The goal here is to give the maximum number of acquired players the most comfortable game experience possible, thus increasing retention and revenue.