Play-testing makes perfect

Play testing is a crucial component to game building.  As we've talked about throughout: game building requires you to make assumptions while you design and build.  Often times you don't exactly know how a system will play out until you build it, but you have to have something defined before you can build it.  This is where testing becomes so crucial.  The basic goal of play testing is to improve the game by watching users play it and record data to evaluate your assumptions and helps you determine what is working and what needs adjustment.  In this article I'll cover how to set up a test, the game builders role and appropriate mindset, what to do with the data collected, and the best way to approach making changes to your game.

Preparation
Before your play-test consider the current state of your game.  If the play test is occuring early in development you will likely have to tell the player missing information, as you are coming closer to the end of development and a tutorial is in place you might not want to tell the testers anything and judge how well they pick up the game without your help.  Whatever stage prepare an introduction for the play testers.  If there are aspects that are missing that will make the experience confusing be tell the testers explicitly.

Have a list of key assumptions that you want to test for.  You will likely glean more information then you are looking for, but it's a good idea to highlight certain features or aspects of your game and ask the users about them directly after the test is over.

It's not a bad idea to have refreshments and possibly food, pizza is a good one, for your play test.  A happy play-tester is a patient play-tester.

During the test
While the session is underway you should remain as hands off as possible.  Observe how users approach and interact with your game.  Are they "getting it"?  Where are things breaking down?  When a user is having trouble don't jump in and help them right away, let them struggle for a bit and try and find their way.  Are they repeating the same actions again and again hoping for  different result?  What lead them to do that?  When they finally do figure out what to do, what was the solution they used to move forward?  If they continue to struggle it may be due to a lack of feedback, indicating what is happening in the game at the present time.  It also could be due to an affordance you set early on that is inconsistent with the current condition.  For instance, if you allowed them to open a red door, and now you have placed a similar red door in front of them but it is just part of the scenery.

Documentation

Write down your findings.  It's a good idea to have a list of questions to ask your play testers once they have finished playing the game.  Ask them if they are "getting" everything.  For example, if you're making a game about clicking different colored shapes ask the testers if they understood what each shape represented or how each shape responded.  You should also set aside time to ask for suggestions.  And they will have suggestions.  Categorize suggestions, identify what the user is commenting on and once you have a list written down, organize it into the different aspects of your game:  art, sound, animation, gameplay, feedback, plot and whatever else might suite your game.  Be sure to take note of everything said, even if something seems ridiculous it might help give an issue or game play a new perspective.

Mindset
Play testers can be brutal and silly.  They can make suggestions that seem to have nothing to do with the games direction and fixate on aspects that you don't feel are important.  It's crucial that you leave you preconceived notions about your game as well as your ego at the door.  Don't try to defend your game, be an objective listener.  Users are giving feed back based on their experience, patience is essential here as well as an ear for how their comments relate to your game.  Keep your mind and ears open.

Try to play test with people in your core user group.  If you are making a game for teen age girls then you'll need to find some teen aged girls.  If you are trying to make a game with broad appeal than you should try to find testers that match that.  Usually you want to keep the test group at a manageable size.  It's best to have not more the three or four people per test administrator.  If you can record the session then do so.  Footage can be very helpful as it's hard to pay attention to every user at every moment and it's easy to miss things.  You can also refer to the footage at a later time.

Don't forget to pay attention to how the are feeling and reacting emotionally.  Facial expressions can tell a lot about how users are experiencing your game.  Hopefully they are excited, if they are showing frustration, be glad you caught it during testing.  Moments of frustration are issues that need to be addressed.

After the test is over
Play-testing is a vital part of the iterative process.  Once you've completed the play test and have documented the results determine the best course of action in moving forward.  You probably had an idea of what you planned to do next but once you compare that with your feed back you might decide to shift course.  Try to have play tests build on one another and compare results from one play test to the next.

Conclusion
Play testing is a vital part of game building.  You should conduct play tests early and often.  Be an objective listener.  Have a set of questions to ask.  Take suggestions.  Categorize your documentation.  React to the play test in your next iteration, then test again.