Debriefing research findings | How to be a games user researcher š¾ šŗ
Research reports | GUR + UX Mentoring | Jimmy Zhou from Riot | An Epic Internship | Playtesting at Develop Conference
User researchers find problems with games. But itās usually someone else who has to fix them.
This means communication is one of the most important skills for a researcher to have - and communicating research findings is critical to having an impact on the quality of games.Ā
In this issue, weāll look at some approaches for sharing research findings, and share some tips from the pros!Ā You can also read this issue in your browser.
Sharing research findings accurately empowers research
From our playtests, we learn how does the game currently differ from the design intent. Much more crucially, we also understand why it differs - what caused players to not understand, or to do the wrong thing.Ā
Understanding the why is essential to making the right decisions about how to fix the problems. Usually, it is someone else who actually makes the final decision on what to do to fix it - a game designer, art director, level designer, etc...
Full and accurate communication of the issues, their causes, and the impacts mean that our colleagues make the right choices.Ā
Making a traditional research reportĀ
The most common way of sharing research findings is to write a report. I have an old example of one on my blog, which - although I would do some things differently today, is still a good idea of what a research report might look like. Iām also sharing a new, real, research report as part of an upcoming GRUX Online talk - look for that in a future issue.Ā
As you will see, the report runs through each of the issues in turn. They are grouped by topic. Matching your groups to āwhat discipline would be working on fixing this issueā can make distributing the findings to the right person easier.Ā
For each issue it explains what the issue was, why it occurred, and what impact it had on the player - this information will help teams make the right fix and prioritise the issue appropriately.Ā
When writing reports, itās important to write concisely, and use plain language - weāre trying to make sure weāre understood. Complicated words donāt impress anyone, and just confuse your readers! Hereās some tips on how to write better.Ā
I would always recommend presenting the report to the team live. People are much more likely to pay attention, will have the opportunity to ask questions if they donāt understand, and it just gets a whole lot more engagement. Although it can be tempting to send a report and say ālet me know if there are other questionsā, I suspect that this leads to reports being ignored, and reduces the chance the team will react to your findings. Ignored findings make running user research pointless, so letās try and avoid that when we can.
A report isnāt right for everyone
Itās really easy to get into the trap of thinking āa user researchers job is to make research reportsā. Our job is to make the game better, and that a research report might not be the best way of doing that.Ā
For some teams, a report isnāt appropriate and would be ignored - instead consider alternative methods of sharing findings:
An interactive workshop
An email covering the top 5 issues
A message on slack
A research insight database that they can interrogate
Access to the raw dataĀ
A conversationĀ
When starting with a team, think about how much time they have, how they communicate currently, and use that to decide what is the right way to share the results so that they donāt get ignored.
Research has multiple audiences
As well as the team who asked for the study, there are some secondary audiences who will also be interested in aspects of your study. They might need a different method for the findings to be shared with them.
Some of those audiences can include:
Executives who donāt need to know the detail, but do need to know if there is a big problem
Other teams working on different projects which some of the findings might be relevant to
Yourself in 12 months time, trying to remember if this study is a topic youāve done before or not.Ā
Many of these wonāt have the time to read through all of the detail of your report - consider some of the other methods that might be more appropriate for sharing research findings with them.Ā Ā
Some prompts to think about:
In interviews, you are likely to be asked about your experience sharing research findings. Some aspects to think about in your answers include:
How did you decide what was the right method of sharing the findings?
How did you understand the needs of the audience and use that to inform your decisions?
How did you evaluate if you had been understood correctly?
Some research results tips from the pros
I asked the community if they had any tips for a successful debrief. Hereās what they said.Ā
Laure described how prioritisation is important for making sure the results are relevant. A common mistake I see (and do myself) is trying to fit in all of the findings, and make them all prominent, when all the team really need is āwhat is the most important thing we need to do nextā - which Laureās method addresses.
For a usability rating scale, I personally like using this severity scale described by userfocus - but otherās exist too!Ā
As Raphael says, understanding your audience is incredibly important. At the start of a project, make sure you know who will be interested in the results, so that you can tailor your delivery to them.
Games Research and UX Design Mentoring
The fantastic team from the IGDA-GRUX SIG have opened up their free mentoring scheme for another batch of applicants.Ā
The mentoring programme has previously partnered over one hundred students with more than fifty industry professionals from top companies such as Sony, EA, Valve, Ubisoft, and Microsoft, and helped many people get their first job in games. It was one of the initiatives I started when on the steering council many years ago, and so Iām really glad itās had a new boost of life with a new team!Ā
If youāre interested in applying, see the mentoring website here (I think the application deadline is the end of September)
User Research and UX Practice ExercisesĀ
Callum Deery is an indie game designer working with Alphabet Soup games, who recognised that itās hard to get in-development builds that can be used to practise research and UX skills at different development stages. Heās working on an RPG and created snapshots, and associated activities, to help people practice.Ā
Activity one: A competitor analysis and usability review of some early menu prototypes
Activity two: Evaluating two potential features and deciding which works better.
Activity three: Helping a team evaluate their demoĀ
Callum would like to thank Kanesha Patterson and Patrick Tan for their feedback so far, and would love to hear your experience with the practice exercises, so do start a conversation with him on twitter about how you get on!
An exclusive interview with Jimmy Zhou from Riot Games
Jimmy Zhou is a user researcher, working on League of Legends at Riot Games. In this exclusive interview, he covers how an internship helped him get into the industry, how to expand your methodology experience, and the importance of starting conversations in the industry.
Read the full interview with Jimmy here.
A quantitative research internship with Epic Games
Ben Taels, UX Research Director at Epic shared an fantastic internship they have available on their team.
The internship looks to require combining survey responses, analytics and other data about users to influence design decisions. Learn more and apply for the job on Epicās site.
Epic are also advertising an entry-level playtest coordinator role, setting up internal and external playtests for Fortnite.
Playtesting at Develop Conference
A busy month ahead for me. Iāll be talking at Develop Conference next month sharing some lessons on running affordable playtesting for teams who canāt afford full-time research support.Ā
Do come and say hi if youāre attending! (and if itās a topic youāre interested in, do sign up for updates on when I publish more resources to help indie teams)
Have a great month and good luck on your games user research journey!
Steve