Pilot Gaming Metrics Analysis

The primary aim of the pilot was to determine if gaming would be embraced by our target end users. This was done through a questionnaire delivered before and after the gaming session. However, as good little scientists we also recorded all of the players inputs during the game session (back-end gaming metrics) and have spent what little free time we have exploring the data to see what else it might be able to tell us about the game’s performance.

Here are some of the interesting connections we were able to make using a combination of the questionnaire and game metric data. Further results are available if you would like to learn more about what we found.

More of our target users are gamer’s than we first thought.

In both the in-country and online pilot, a greater proportion of participants had experience with computer games than those with no experience.

gamers v non gamers pilot

Their gaming experience didn’t have a big impact on their performance in the pilot game.

We used several indicators to measure their performance:

  • Proportion of bioassays completed
  • Proportion of susceptibility reports viewed
  • Proportion of times insecticide rotations were implemented
  • Proportion of players who looked at health info for district 3 in round 3 *
  • Proportion of players who chose the correct intervention for district 3 in round 4
  • Score*

*Online pilot only

(Proportion who looked at health info for district 3 in round 3 not included in graphs below as there was not enough data to run statistical analysis)

gamer v non gamer bioassay

The only indicator that displayed a significant difference was the number of bio-assays conducted between gamers and non-gamers.

gamer v non gamer report + rotategamer v non gamer CI +score

The players learnt to conduct more bio-assays as the game progressed.

The pilot game had 4 rounds in total, the first round was a tutorial where the player was guided through the functionality in the game. All players had to conduct bio-assays on at least one insecticide to complete the round, which explains why more players conducted bio-assays in round 1 and then much less in round 2.

p[erformance by age

Looking at reports after generating them was obvious to the player.

looking at reports

Players had to access the games database to view susceptibility and abundance reports, what is impossible to determine from this is did they do it to make an informed decision or did they simply work out that their score increased by clicking on the reports.

The players age had some impact on their performance in the game.

By looking at both the proportion of bio-assays conducted and number of reports accessed we can see that the performance dropped as the age increased. We believe the bar displaying the performance of players aged 56+ in as anomaly due to the low sample size of that age group.

peformance by age

Displayed in this post is just some of the analysis of the pilot game metric data. Analyzing this data has taught us a lot about connecting indicators with the learning objectives right at the start of the development process, and to ensure we collect the right data as soon as the game is rolled out. It has also taught us that we need to be considerate of our target users demographic when designing the game, but equally we should not underestimate them.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s