Debriefing research findings | How to be a games user researcher 👾 📺
Research reports | GUR + UX Mentoring | Jimmy Zhou from Riot | An Epic Internship | Playtesting at Develop Conference
User researchers find problems with games. But it’s usually someone else who has to fix them.
This means communication is one of the most important skills for a researcher to have - and communicating research findings is critical to having an impact on the quality of games.
In this issue, we’ll look at some approaches for sharing research findings, and share some tips from the pros! You can also read this issue in your browser.
Sharing research findings accurately empowers research
From our playtests, we learn how does the game currently differ from the design intent. Much more crucially, we also understand why it differs - what caused players to not understand, or to do the wrong thing.
Understanding the why is essential to making the right decisions about how to fix the problems. Usually, it is someone else who actually makes the final decision on what to do to fix it - a game designer, art director, level designer, etc...
Full and accurate communication of the issues, their causes, and the impacts mean that our colleagues make the right choices.
Making a traditional research report
The most common way of sharing research findings is to write a report. I have an old example of one on my blog, which - although I would do some things differently today, is still a good idea of what a research report might look like. I’m also sharing a new, real, research report as part of an upcoming GRUX Online talk - look for that in a future issue.
As you will see, the report runs through each of the issues in turn. They are grouped by topic. Matching your groups to ‘what discipline would be working on fixing this issue’ can make distributing the findings to the right person easier.
For each issue it explains what the issue was, why it occurred, and what impact it had on the player - this information will help teams make the right fix and prioritise the issue appropriately.
When writing reports, it’s important to write concisely, and use plain language - we’re trying to make sure we’re understood. Complicated words don’t impress anyone, and just confuse your readers! Here’s some tips on how to write better.
I would always recommend presenting the report to the team live. People are much more likely to pay attention, will have the opportunity to ask questions if they don’t understand, and it just gets a whole lot more engagement. Although it can be tempting to send a report and say “let me know if there are other questions”, I suspect that this leads to reports being ignored, and reduces the chance the team will react to your findings. Ignored findings make running user research pointless, so let’s try and avoid that when we can.
A report isn’t right for everyone
It’s really easy to get into the trap of thinking ‘a user researchers job is to make research reports’. Our job is to make the game better, and that a research report might not be the best way of doing that.
For some teams, a report isn’t appropriate and would be ignored - instead consider alternative methods of sharing findings:
An interactive workshop
An email covering the top 5 issues
A message on slack
A research insight database that they can interrogate
Access to the raw data
A conversation
When starting with a team, think about how much time they have, how they communicate currently, and use that to decide what is the right way to share the results so that they don’t get ignored.
Research has multiple audiences
As well as the team who asked for the study, there are some secondary audiences who will also be interested in aspects of your study. They might need a different method for the findings to be shared with them.
Some of those audiences can include:
Executives who don’t need to know the detail, but do need to know if there is a big problem
Other teams working on different projects which some of the findings might be relevant to
Yourself in 12 months time, trying to remember if this study is a topic you’ve done before or not.
Many of these won’t have the time to read through all of the detail of your report - consider some of the other methods that might be more appropriate for sharing research findings with them.
Some prompts to think about:
In interviews, you are likely to be asked about your experience sharing research findings. Some aspects to think about in your answers include:
How did you decide what was the right method of sharing the findings?
How did you understand the needs of the audience and use that to inform your decisions?
How did you evaluate if you had been understood correctly?
Some research results tips from the pros
I asked the community if they had any tips for a successful debrief. Here’s what they said.
Laure described how prioritisation is important for making sure the results are relevant. A common mistake I see (and do myself) is trying to fit in all of the findings, and make them all prominent, when all the team really need is ‘what is the most important thing we need to do next’ - which Laure’s method addresses.
For a usability rating scale, I personally like using this severity scale described by userfocus - but other’s exist too!
As Raphael says, understanding your audience is incredibly important. At the start of a project, make sure you know who will be interested in the results, so that you can tailor your delivery to them.
Games Research and UX Design Mentoring
The fantastic team from the IGDA-GRUX SIG have opened up their free mentoring scheme for another batch of applicants.
The mentoring programme has previously partnered over one hundred students with more than fifty industry professionals from top companies such as Sony, EA, Valve, Ubisoft, and Microsoft, and helped many people get their first job in games. It was one of the initiatives I started when on the steering council many years ago, and so I’m really glad it’s had a new boost of life with a new team!
If you’re interested in applying, see the mentoring website here (I think the application deadline is the end of September)
User Research and UX Practice Exercises
Callum Deery is an indie game designer working with Alphabet Soup games, who recognised that it’s hard to get in-development builds that can be used to practise research and UX skills at different development stages. He’s working on an RPG and created snapshots, and associated activities, to help people practice.
Activity one: A competitor analysis and usability review of some early menu prototypes
Activity two: Evaluating two potential features and deciding which works better.
Activity three: Helping a team evaluate their demo
Callum would like to thank Kanesha Patterson and Patrick Tan for their feedback so far, and would love to hear your experience with the practice exercises, so do start a conversation with him on twitter about how you get on!
An exclusive interview with Jimmy Zhou from Riot Games
Jimmy Zhou is a user researcher, working on League of Legends at Riot Games. In this exclusive interview, he covers how an internship helped him get into the industry, how to expand your methodology experience, and the importance of starting conversations in the industry.
Read the full interview with Jimmy here.
A quantitative research internship with Epic Games
Ben Taels, UX Research Director at Epic shared an fantastic internship they have available on their team.
The internship looks to require combining survey responses, analytics and other data about users to influence design decisions. Learn more and apply for the job on Epic’s site.
Epic are also advertising an entry-level playtest coordinator role, setting up internal and external playtests for Fortnite.
Playtesting at Develop Conference
A busy month ahead for me. I’ll be talking at Develop Conference next month sharing some lessons on running affordable playtesting for teams who can’t afford full-time research support.
Do come and say hi if you’re attending! (and if it’s a topic you’re interested in, do sign up for updates on when I publish more resources to help indie teams)
Have a great month and good luck on your games user research journey!
Steve