The Chaos Factor: The Biggest Challenge for Game Analytics?

How do you generate order from chaos? This is fundamentally the challenge game analysts face when given the task of making sense of some of the most complex information systems in the world: Massively Multi-Player Online Games. In this post we broach the topic of game balance in MMOGs and why making it perfect is so hard.   

In MMOGs, up to millions of players interact across up to hundreds of servers running instanced versions of the same game world, and the games can have a lifetime of several years. These games form some of the most complex computer systems in the world not only due to the raw computational and infrastructural challenges, but notably because they involve humans – lots and lots of humans. And if there is one thing we know as developers, it is that humans do what we expect them to do.

When providing a persistent service, it becomes essential to be able to monitor the ongoing changes in user population, and to be able to detect and react to any undesirable changes in its the behavior. Monitoring can prevent player attrition or unstable in-game economies, as well as help in detecting fraudulent behavior.

One of the key components of user population monitoring and evaluation processes in MMOGs is balance analysis, i.e. the evaluation of the expression of the game mechanics as they meet the users.

Essentially, balance design is the practice of tuning game mechanics and rules in order to avoid system inefficiencies or undesirable behavior impacting the player experience. An unbalanced game runs the risk of providing substandard player experiences, e.g. through being too easy or too difficult to play, which in turn can lead to loss of potential revenue.

The system’s balance in digital games is not necessarily meant to be fair to the players, but rather unfair, which when managed well leads to engagement and excitement in e.g. action games. One of the main goals of balance analysis is to evaluate whether the game successfully supports all the player strategies intended by the games´ design.

Balance design and analysis can be achieved in a number of ways, using e.g. statistical analysis, randomization, feedback loops and simulation. The more mechanics are present in the game, and the higher the number of possible different player actions, the more complex the interactions between the components of the game system, and thus potentially the more challenging is both the process of designing such systems and subsequent analysis.

Let’s look at an example from a typical MMORPG:

Already by now we have more possible combinations of attack and defense than there are stars in the Milky Way and we would have to be moving rapidly beyond quantum computing capacities to accurately model the entire complexity of the system. This means we have to either ”cheat” and make restrictions on interactions and/or release unfinished systems and change them on a running basis via player feedback and analysis. Both approaches are useful to solve the fundamental complexity problem.

Striving for perfect balance: change

There are two sources of change in MMOG population behavior:

1) Changes instigated by the population (or part of it)

2) Changes instigated by altering game design.

The latter generally leads to the former, e.g. releasing a patch which alters the abilities of the player characters (or similar in-game representation), leads to a corresponding change in how the users play the game as some modes of play become unviable and new modes open up.

A stable example is changes to player character classes, e.g. in games like World of Warcraft, Rift and Lineage, which leads to players adopting new strategies because their characters can now be optimized in new ways. Conversely, players may develop strategies not foreseen by the designers of the game, which unbalance the game, e.g. make certain character classes very powerful in specific contexts, which can lead to player dissatisfaction (those who have chosen to play the less powerful classes). This in turn can require changes in the game system in order to restore the power balance between character classes. In this context, “game system” refers to all mechanics that relate to the players – which means the majority of the rules present in the typical MMOG.

It remains a debated topic whether striving for perfect balance in MMOG design is the best way of handling the raw complexity of contemporary persistent-world games. Proponents of imbalanced systems have for example suggested that such games would lead to interesting gameplay. Another point raised in the debate is the “chaos factor” argument in MMOG design, which basically states that it is impossible to pre-model everything due to the size of the possibility space, and that game balance based on pure data analytics techniques fail when the game meets the player community.

For example, if a patch is applied that slightly modifies the abilities of a single character class, this can have a cascade effect into the balance of all character classes and artificial entities, which given variability space in modern MMOG can mean needing to consider the effect of hundreds of variables.

If the goal is a perfectly balanced game, this suggests that the choices made during character creation (or the equivalent in MMOGs that do not use traditional avatar-based 3D characters, such as EVE Online) should lead to equal opportunities and power balance, which in turn hopefully means an equal opportunity for all players to have a fun and engaging experience. From this point of view, a game is balanced if perfectly played characters are equally effective when weighted across all the character abilities in the game.

Given the number of character abilities, levels, combinations of traits, different types of opponents etc. in a contemporary MMOG situation, a huge variability space is thus produced where balancing becomes a challenging enterprise. This is exemplified by the reverse-engineering of dozens of formulae underlying the MMORPG World of Warcraft by the player community.

Irrespective of which underlying philosophy towards balancing MMOGs and especially MMORPGs is adopted, balancing such game systems is regarded as one of the hardest challenge in the game industry.

Having discussed some of the general details, in future posts we will go into more detail with balance analysis – stay tuned.



Game Analytics is a cloud hosted solution for tracking, analysis and reporting of game metrics. We will improve quality assurance, bug fixing, game design and monetization of games. Drop us a line on to learn more!

About the author

Anders Drachen

Anders Drachen is a veteran Data Scientist and Lead Game Analyst at Game Analytics. His work in the game industry as well as in data and game science is focused on game analytics, business intelligence for games, game data mining, game user experience, industry economics, business development and game user research. He is one of the most published experts worldwide on the topic of game analytics, user research, game data mining, and user profiling, having authored more than 60 research publications on game analytics, user testing, and business intelligence in game development, and co-edited the first book on the topic.


Third Party Analytics: what are the options?

Outside the companies that actually develop computer games and other forms of interactive entertainment, a rapidly increasing number of third-parties have emerged in the past few years to provide analytics-related services to companies. Here we provide a brief overview of the types of providers available and the pros/cons of their services.

Fundamentals, Game Data Mining

The Allure of Numbers

For people working with game analysis, gameplay metrics (measures of player behavior, e.g. navigation, item- and ability use, trading etc.) is the kind of data that – if data were gemstones – would be diamonds. And not the regular white, discount version but the flawless blue ones that we need Indiana Jones to steal back from bad-tempered natives in the South American jungle.

Game Data Mining