Pilot Gaming Metrics Analysis

The primary aim of the pilot was to determine if gaming would be embraced by our target end users. This was done through a questionnaire delivered before and after the gaming session. However, as good little scientists we also recorded all of the players inputs during the game session (back-end gaming metrics) and have spent what little free time we have exploring the data to see what else it might be able to tell us about the game’s performance.

Here are some of the interesting connections we were able to make using a combination of the questionnaire and game metric data. Further results are available if you would like to learn more about what we found.

More of our target users are gamer’s than we first thought.

In both the in-country and online pilot, a greater proportion of participants had experience with computer games than those with no experience.

gamers v non gamers pilot

Their gaming experience didn’t have a big impact on their performance in the pilot game.

We used several indicators to measure their performance:

  • Proportion of bioassays completed
  • Proportion of susceptibility reports viewed
  • Proportion of times insecticide rotations were implemented
  • Proportion of players who looked at health info for district 3 in round 3 *
  • Proportion of players who chose the correct intervention for district 3 in round 4
  • Score*

*Online pilot only

(Proportion who looked at health info for district 3 in round 3 not included in graphs below as there was not enough data to run statistical analysis)

gamer v non gamer bioassay

The only indicator that displayed a significant difference was the number of bio-assays conducted between gamers and non-gamers.

gamer v non gamer report + rotategamer v non gamer CI +score

The players learnt to conduct more bio-assays as the game progressed.

The pilot game had 4 rounds in total, the first round was a tutorial where the player was guided through the functionality in the game. All players had to conduct bio-assays on at least one insecticide to complete the round, which explains why more players conducted bio-assays in round 1 and then much less in round 2.

p[erformance by age

Looking at reports after generating them was obvious to the player.

looking at reports

Players had to access the games database to view susceptibility and abundance reports, what is impossible to determine from this is did they do it to make an informed decision or did they simply work out that their score increased by clicking on the reports.

The players age had some impact on their performance in the game.

By looking at both the proportion of bio-assays conducted and number of reports accessed we can see that the performance dropped as the age increased. We believe the bar displaying the performance of players aged 56+ in as anomaly due to the low sample size of that age group.

peformance by age

Displayed in this post is just some of the analysis of the pilot game metric data. Analyzing this data has taught us a lot about connecting indicators with the learning objectives right at the start of the development process, and to ensure we collect the right data as soon as the game is rolled out. It has also taught us that we need to be considerate of our target users demographic when designing the game, but equally we should not underestimate them.

Advertisements

Where it all started…

When we first started our exploration into the use of serious gaming for vector control we had no idea what impact it would have, or if anyone would even want to play our games. The reception our pilot game received and the buzz it generated in the vector control community was more than we could have ever imagined.

The potential of gaming can’t be denied. Now we need to work together to make it a reality.

Game Model Prototype

Our mathematical main man Andy South has been hard at work developing a ‘game model’ which will drive the behavior of the virtual vector population in the Resistance game. An early prototype demonstrating the ‘game model’ can be found HERE.

Model prototype

Warning – imagination is needed to picture how this will eventually work in the game. It currently demonstrates how player input and different game environments might effect the vector abundance and resistance levels. How this will end up being visually represented and the game mechanics that will drive it is what we are having fun with now.

If you have any questions or feedback for Andy please get in contact. charlotte.hemingway@lstmed.ac.uk

Hello again!

Rather than bombard your email accounts with endless updates about all the exciting activities going on. We’ve decided to create an online space. Where you can pick and choose the ETCH news that’s important to you.

We’ll be posting regular updates on the development of our Resistance Game; related news articles on innovative ways people are educating and communicating in the public health world; and anything and everything ETCH including new projects and ways you can get involved.

Together we hope to improve the lives of many through the delivery of innovative communication and learning tools.

“Video Games are bad for you? That’s what they said about rock and roll.” – Shigeru Miyamoto