Find the fun - measuring enjoyment in games user research [How to be a games user researcher] 👾 🔍
Measuring fun in games | Patrick Tan's journey to research at Hothead Games | Jobs with WB, Kabam, Epic & Activision | Free Games Research & UX Conference
It seems sensible for developers to want to know if their game is fun. Games are entertainment - and so a measure of whether the game is working would be “do people have fun playing it”.
This means that researchers get asked this question reasonably frequently. Today we’ll look at some of the challenges with answering it, and how researchers can handle measuring fun.
Read on to learn more - or open this issue in your browser.
Finding the fun in games
It’s not hard to come up with a question about fun. You can ask something like…
Or you can ask…
Or even...
Each of these variations gets you a fine answer, and it can keep researchers busy for hours debating exactly which is the most reliable question wording to use. (My own take is that it doesn’t hugely matter as long as you consistently ask it in the same way when doing comparisons).
And that’s great - you now have a number to represent fun (or enjoyment, or how good the game is).
But what does a number mean?
A number by itself can not be that helpful. “This game is eight fun” only inspires more questions, including:
“Is eight good?”
“What about it makes it eight?”
“Why wasn’t it nine?”
I suspect people hope this is predictive of critic review scores, but in my experience that has never been the case - what game reviews are assessing on is completely different from what players are able to assess on.
The number is also heavily biased by the context in which you’ve asked it - especially for high profile games. You’ve brought in some people to play an unreleased game, in a secret user research lab, which is a fun experience. You’re learning that they’ve had an eight of ten day out, not that they have played an eight out of ten game.
Using the number as a benchmark
Some value is in using the number to compare to other things. Eight out of ten is meaningless by itself, but when you know another experience is seven out of ten, that now has meaning - this one is better.
Things you can compare to include:
The ‘score’ for one level vs another level.
The same game at different stages of development
Other prototypes or games at a similar stage of development.
We covered how to do the maths to compare things in a previous newsletter.
Keep in mind though that players will be rating everything about the experience they had, which can make comparison difficult. If comparing your game with something at a different development stage, the more ‘finished’ one will probably rate higher - which is only telling you it’s more finished, not that it’d be the better experience once complete.
As mentioned above, players will also be rating if they are having a fun day out - something to watch out for if you don’t run the test in the same location using the same method, or if you use a different type of player. It’s easy to come to misleading conclusions.
Working out what is fun
I find a lot of the value of asking for a ‘fun’ rating is in the discussion after with players. It can lead to learning why they rated it high or low, which is a lot more useful for game teams than an overall score. Asking ‘why you gave that score’ helps teams learn what elements are working, and what players believe are the biggest issues with the game currently.
How I handle fun scores
How I treat measuring fun is:
Use one of the questions above to get a numerical rating
Ask some interview questions to understand ‘why they gave that score’
Do the maths and make a graph - either giving the number in isolation, or as a comparison to something relevant
Combine the graph with the qualitative comments about why people gave the scores, grouped by ‘good things they said’ and ‘bad things they said’
Put this all at the end of the report, after behavioural and usability findings, and explain why it’s in a deprioritised section by itself
Explain to my game teams that the number is difficult to interpret, because of the influence the study design and development stage has on the score. Give extra caveats if it’s a qualitative study where we have a low number of participants. Contrast it to the behavioural findings earlier in the report which we think are safer to take action on.
Or push for better measures
I asked peers for their opinions on fun scores.
Amanda Jarrell’s approach is a really sensible way of handling this. Working closely with development teams to push them to more accurate measures is a more effective way of judging if we’re creating the player experience we want to create.
One way of achieving this is getting teams to define the emotional experience they are trying to create, and then forming questions asking about those emotional experiences, rather than overall fun. This unfortunately blurry picture of the emotional map of Journey is a great example of defining an emotional experience of a game so it can be measured (source: The Dead Pixels). The Guardian newspaper wrote more about emotional game design and mapping emotional arcs here.
This could also include pushing teams to define more objective measures than emotion. How many times do you want players to fail this challenge and How long do you think this section should take players are a lot easier to measure and handle.
Looking for more on fun? Do read John Hopson’s article ‘Why I hate fun’ for a great critique of ‘fun scores’ and a description of some alternative approaches.
Also this month there was some great fun score related discussion in the replies to this tweet by Elizabeth Zelle - read the tweet and the replies.
Some prompts to think about:
What are the risks from sharing a ‘fun’ score?
What questions should you ask teams who want to run a test to discover “is my game fun?”
For a recent game you’ve played, what emotional experience is it trying to create? How would you measure if it’s successful?
Patrick Tan’s Journey Into Games User Research
This month I’ve been talking to Patrick Tan, User Researcher at Hothead Games about his journey into games user research, and getting his first role in the games industry.
He shared his experience applying for roles, getting the experience necessary to work in games and top tips for finding entry level games jobs.
Read the full interview with Patrick here.
Entry Level Games User Research Jobs
Spotted some great opportunities this month, including:
Warner Brothers - Temporary User Research Assistant
Epic Games - Contract UX Analyst
Activision - User Research Internship
If you decide to apply, let me know how you get on!
Games Research & UX Conference in November
GRUX Online is a free one day conference on user experience and user research in video game development, running on the 19th November. This year I am one of the sponsors of the conference, and am really excited about the programme the team has put together, including:
An all star panel talking about how to get started in Games UX
Morgan Baker, Améliane Chiasson and Tara Voelker on making horror games accessible
Elise Torfs covering how they establisher user research at Remedy
(and many other excellent talks. See the full programme)
I am speaking at the event about how to get games user research experience before you have a job in the industry. I’m confident that the approach I describe will help you stand out when applying for your first games user research job.
Come and join us for the free conference on the 19th November.
Learn how to be a games user researcher for Christmas
The christmas rush is shortly upon us - the perfect time to add the How To Be A Games User Researcher book to your Xmas list.
The book is an practical guide to how games get made, how to run user research studies on games and how to get the experience needed to excel at job interviews. People have said lovely things such as :
“An excellent book on games user research! This is great for those breaking into the field or starting laboratories, all the way to field veterans who like to have a handy reference on them.”
“If you are considering breaking into games user research, or looking to refresh your memory on important aspects of your career, I'd highly recommend picking up this book. “
“This book explains with clear, engaging writing the concepts, procedures, and soft skills that an aspiring professional games user researcher has to master in order to understand how players feel about a game (or a part of it) and translate those feelings into useful data to the rest of the development team.”
If you’re looking to start your first job in the games industry in 2022, this is the perfect time to pick up the book!
Learn more about How To Be A Games User Researcher on Amazon.
See everyone next month for the next deep dive into games user research.