Apr 29, 2013
Have you ever worked with a title for months in a row? Do you known every bush, button and sound effect by heart? Did you unwittingly become blind to its flaws? If you are a games industry professional, I assume that you have.
Knowing your game in and out, from splash screen to rolling end credits, is great – but also dangerous. How can you find flaws with the interface when you know how every single button works? How can you balance difficulty when you can beat the final boss on Hard by rolling your face on the keyboard?
Either you have to become even more tremendously awesome at what you do and learn to always see your game from thirty different angles at once, or you make use of the wonderful tool that is focus testing. Or, as you may call it, playability and usability testing. (Those are the terms we commonly use at the office – there might be other terms for the same thing, but I hope you understand what I mean.) You may already be familiar with this type of testing, or you may not. Either way, I hope you find this post interesting.
For both playability and usability testing, the principle is simple. Find someone that you don’t know, who’s never seen the game before, and watch them as they play. It should ideally take place in a home environment, and no one besides the tester should be physically present in the room – video recording, eye tracking, heart rate monitors and so on should be used instead. Developers are never allowed to participate in any way, in order not to get the testers ‘star struck’. For the average QA team, however, it might be hard to achieve all of this. There are QA studios you can hire to do exactly these types of testing, but you may also skip the rules you can’t follow and carry out the test the best you can. You can usually get quite interesting results anyway.
Let’s look at the two sub-types of testing I mentioned above one at the time. Playability testing revolves around the actual gameplay. Do the test subjects understand how to finish the puzzle in level 5? Is the text in the tutorial too long? Is the game engaging? Do the players get stuck anywhere? Note that if they do, you are not allowed to help them if they have not specifically asked for it – and even then you should wait five minutes, so that they really have to try and get it right on their own.
Usability testing revolves around the interface. The testers might customize the loadout or use the diplomacy screen. Is the difference between a slim, fast football player compared to a big, strong one clear enough? What do the players think about the options screen? I’m sure you can come up with a lot of questions that applies to your game.
When the test subjects have played for a while, they should fill out a form with questions about their experience. The observers should talk for a short while with each other about what they have noted during the session, and then both test subjects and observers should discuss the game together. Finally, the observers should write a report including all observations, opinions, issues and last but not least suggested improvements for the game and hand it to the developers.
I strongly recommend that everyone – small indie and AAA studios alike – make use of focus testing at least once during the development of a game. I have never participated in a test of this kind that hasn’t uncovered a good handful of serious issues. You might find your own variation that suits you better, but the main features, such as testing on people that have no relation to the title, should be there. This blog post really only scratches the surface, but I hope I have opened a few eyes and made you want to know more – hit Google for more information!
I’ll leave you this week with an anecdote: In one of the episodes of the fantastic Extra Credits over at Penny Arcade, they tell the story of how James Portnow once tried a game, in which he just could not understand how to open the inventory. In the end, he has to ask the developers. Apparently, you had to triple click the avatar – which they though was obvious, since they were so used to it, but which understandably made no sense to James what so ever. This is a good example of how focus testing works – the faulty designs you take for granted is easily discovered by a user who looks at the game with fresh eyes.
And with that, I will say my goodbyes.
Be seeing you!
– The Secret Tester