Find usability issues in games with playtests [How To Be A Games User Researcher š® š° ]
How to find games, usability issues | EAās Jess Tompkins | Write a CHI paper | Junior UX + user research jobs | free stickers & moreā¦
Usability issues can ruin games.
When players canāt understand what they are meant to do, where they are meant to go, or what is happening, they get confused, bored and ultimately drop out. This impacts their opinion of the game, retention, and creates an experience that just isnāt fun.
User researchers can help.
One of the core skills for researchers, and one of the focus areas for the How To Be A Games User Researcher book is how to find and deal with usability issues. This helps make games that players understand and enjoy.Ā
Today weāre going to look at how to use playtests and UX research studies to find usability issues in games. We also have an exclusive interview with EAās Jess Tompkins, a course for writing CHI papers, new junior user research jobs, free stickers & moreā¦
š (Plus the discount for the Playtest Kit is still running till the end of the month - use code HTBAGUR to get 20% off!) š
Read on to learn how to find games usability issues, or open this issue in your web browser to read it later
Find games usability issuesĀ
Today weāre looking at usability issues. What we mean by usability is ācan players do what they are expected to doā.Ā
Thatās different to opinions about a game. āI donāt like that the weapons break in Zeldaā is an opinion.Ā āI donāt understand why my weapons keep breakingā is a usability issue. Today weāre tackling the latter.
1. Start with understanding the design intent
Usability issues are āwhen the player canāt do what we expect them to doā. To spot that, we need to know āwhat do we expect them to doā.
This involves talking to our colleagues. Designers have an idea about how they expect the game to work, and what they think players should understand and do. You need to learn what their expectations are (the ādesign intentā).
Exactly who to speak to, and what to ask depends on what youāre testing - which is why our studies start with scoping around research objectives.Ā You may need to talk to combat designers, UI artists, level designers, producers, or others, But for an idea about the range of things we may need to understand look at Liz Englandās āThe Door Problemā. You need to know what the designer would say in answer to every one of those questions.Ā
This can be a lot of chatting.
But it will make the other steps a lot more effective!Ā
2. Create the right context to create realistic issues
Next comes creating the task. What are we actually going to ask the player to do.Ā
Some tasks can be very broad āComplete this levelā or āplay for thirty minutesā. Sometimes they can be very specific āCraft an upgrade to your horse armourā or āPlace this object on the shelfā. Exactly what level to set your task depends on your research objectives, but my preference is to lean towards broader tasks where possible, because it allows unexpected issues to emerge.
If you are live in your playtest, you can be looser on the task. Because you are in the room, and watching them play, you can jump in with an improvised task at the right time.Ā
Designing tasks in UX tests can be tricky. Revealing too much information will artificially lead players to the solution. Revealing not enough information (such as missing tutorials) will create artificial situations that wonāt occur in the final game. This needs a lot of thought to balance.
My preference is to be as undirected as possible, while meeting the research objectives. āPlay through this bitā is fine, as long as they are exposed to the right content, any tutorials or pre-requisite content, and is easier when you are on hand to subtly guide players towards the right content.
3. Be quiet, watch and look out for deviations
Every time you intervene with players, you are introducing artificial elements to the playtest. Unless you intend to answer questions live for every player post-launch, you should resist answering questions, or revealing information in the session.Ā
Instead your attention should be focused on listening and watching. Observe what players are doing in the game, and when they do something unexpected, write it down. Looking back at your research objectives, and a good understanding of the design intent will help you recognise unexpected behaviour.Ā
Write down what you see (weāll cover good note taking in a future issue).Ā
When necessary, hand out new tasks to direct players attention to the right areas. (but still be very careful that the words you use or instructions you give donāt reveal information that the player wouldnāt otherwise have)
4. Ask questions to understand what youāre seeing
Observed behaviour is only half of the story. In order to understand why that behaviour occurs, we need to know whatās happening in playerās heads.
This requires asking them questions.Ā
As described above, there is a massive risk that your questions reveal information to the player that they wouldnāt normally have - artificially influencing your playtest. Bland, non-leading questions are required - such as āwhat is happening nowā. We wrote more about how to ask good questions in a previous issue.Ā
5. Use the impact to rate severity
Itās common to spot a lot of problems, and as covered in the How To Be A Games User Researcher book, researchers go through a structured analysis process to uncover and describe them all.
In order to focus the teamās attention, we need a process for identifying which problem is most pressing. We do this by anticipating what issues will be most disruptive to players when they play.
This extract from How To Be A Games User Researcher explains one approach for doing this:
A method I like is a four-point scale for issues:
Critical
High
Medium
Low
Each issue starts as āLowā. Then ask three questions about the issue. Each time the answer is yes, raise the severity up a level.
These questions are:
Is this something that the player needs to do to progress?
Did the moderator need to step in to help resolve the issue?
Once the player had overcome the issue, did they know how to avoid it when they encountered it again?
When running your playtest, noting āwhat happened to the player as a result of this problemā, gives you the raw information you need to do this analysis + rating process. We will cover analysis in more depth in future issues.
A chance to practice
Iām sharing a session of a player who is playing Pokemon Unite for the first time. During their play session, they encounter some usability issues with the game.
Watch an unmoderated usability test of Pokemon Unite
As you watch, think about:
The design intent (what did the game team expect players to understand or do)
The issues the players encounter
What questions youād ask, if the session was live
What impact the issues had on the playerās experience
(Feel free to share what you spot with me on Twitter!)Ā
Other games user research news
EAās Jess Tompkins on PhDs, Inclusion & Research Impact
I interviewed EAās Jess Tompkins about her role as a Senior User Researcher. She shared her own journey into games, the value of doing a PhD, challenges around inclusion and diversity in games, and the importance of judging research on itās impact.
Read the full interview with Jess Tompkins ā”ļø
Junior Games User Research Jobs
Mojiworks is a Snapchat game developer, and looking for an associate user researcher in the UK
Warner Brothers are looking for a User Research Assistant
This PhD from the Intelligent Games and Game Intelligence Centre is being run in collaboration with Creative Assembly. One of the roles is about using AI in playtesting, which is an interesting topic!Ā
Nintendo are hiring an intern to look at the UX of their websites.Ā
Activison are hiring UR moderators. Hereās some guidance from their User Research Director, Adams Greenwood-Ericksen that he shared in Discord.Ā
Hey all, we're opening up our yearly search for new UR moderators. Take a look and apply if you're interested. We don't expect any previous experience in the industry for these roles. Word of advice: we get a lot of applications from folks who aren't that into games and just want a UX role in tech - I'd suggest you make sure your resume has a "favorite games" section or indicates your interest in GUR specifically in some other way! Feel free to DM me with any questions.
Write better academic research papers
I know many people in our community are still in academia. You may already know Lennart Nacke, a recognised expert in the field of academic games research.Ā
Heās launched a new course for academics and industry professionals who want to learn how to get papers accepted and published at prestigious conferences like CHI.Ā
Lennart has kindly given us a 50% discount code to share, to get the course of 20+ video lessons, and templates. Use the below link to automatically apply the discount.
Get the discount on āHow To Write Better Research Papersā today š
A day in the life of a games user researcher
You may already be familiar with the GUR Cafe podcast, which regularly covers important gams user research topics, and exposes the real experience of researchers working in games.Ā
I wanted to recommend their recent episode āa day in the life of a GURā where Ubisoft Montrealās Tristan Carreiro-Blanc & Gabrielle Mourra share what working with game developers, and running studies is really life - incredibly helpful for people looking to understand what the job is really like.
Find all episodes of the podcast here š§
Free games user research stickers
I have some extra GUR stickers! I also want to refresh the testimonials I have introducing the newsletter, to help people recognise if itās appropriate for them. (see the great ones from Lucy and Natali here)Ā
Hereās the deal. For the first 25 people who do so, if you tweet a short testimonial about the newsletter, Iāll mail you a free sticker - entirely free of charge.
Remember to mention my twitter ID somewhere in the message, so that I see it!Ā (@steve_bromley). Iāll then DM you to organise sending you the sticker.
Better playtesting today.
Thanks for the fantastic reception to the Playtest Kit so far. This is probably the last warning Iāll give about the launch discount running out imminently, so itās the best time to snap it up.
The kit aims to address issues that developers have that stop them running better playtests including:
Working out the best time to run a playtest
How to make sure you learn new and relevant things
How to find the recruit kind of players, and schedule them easily
How to write good questions and design good tasks
How to write a reliable survey
How to interpret messy data and draw conclusions
š Learn more, and get the Playtest Kit today at playtestkit.com (use the discount code HTBAGUR to get 20% off during May) š
Thatās it for today. Have a great month!Ā