What is a “bad playtest”?
by Gil Hova
One sentence I hear from game designers that I’m not always on board with is, “I had a bad playtest.”
What happened? Maybe you spent a long time preparing a radical new change to the game. Maybe a new playtester tried a completely unexpected strategy. Maybe this is a new game’s maiden voyage.
In any event, no one had fun and you had to cut the test short.
Is that a bad playtest? I’m not convinced it is.
They’re tough playtests, sure. I would love all my playtests to end in wild whoops and cheers, and people telling me, “Gil, you’ve done it again. I expect this game to win the SdJ, a Pulitzer, and a Nobel.”
But we all know that doesn’t happen. Sometimes you have to stop the playtest once you see the game state has stalled. Other times the game lurches to an end, and no one wants to be the first to talk.
These are tough playtests. But they’re not bad playtests. They’re necessary playtests.
These are the playtests you need to have if you want your game to grow. Every game has been through them. You have the patient open on the table. Don’t be squeamish now; go in there and find out what’s wrong.
Did the game stall? Why? Did players run out of money? Were they always taking the same actions? What’s the interesting thing you want them to do? What would have incentivized them to do those interesting things?
Did one player run away with the game, or fall helplessly out of competition early on? Was there an action she did early on that made her unbeatable or made the game unwinnable? What was that action? Was there something the other players could have done to prevent it, or something she could have done to pull herself out of the situation? Does your game need a negative feedback loop to rubberband players’ scores?
Were players simply not engaged? Did they not see their available choices? Were things too opaque? What can you do to make your mechanisms more transparent?
These are all examples of difficult, tooth-gritting playtests I’ve had for Prolix, Battle Merchants, and Prime Time. I needed all of them to figure out fundamental problems with the game, and learn their root causes. Once I acknowledge that the game isn’t going swimmingly, I start probing why players aren’t having as good a time as they could.
The main question to ask during a difficult playtest is: what should the game be encouraging to be doing (or forbidding them from doing)?
It’s tough at first, especially if this is your first game. These games are your darlings, and you have to make the painful acknowledgement that there is a deep, serious flaw with that darling.
But once you do that, the process gets surprisingly easier. Most playtesters will enthusiastically tell you what went wrong for them. Some of them will suggest fixes. Listen to everything. You don’t have to implement all the feedback; in fact, at times, the players may be suggesting fixes to a problem that you can more elegantly fix with your perspective (for example, check out this wonderful post by Daniel Solis that describes just that).
Okay, so with that said… is there such a thing as a “bad playtest”?
Well, of course there is. Here are a few bad playtests I’ve had…
Example 1:
We finish playing. There is silence. I ask, “What did you think?” Everyone says, “It’s okay. Not bad.” I try to get more feedback out of the playtesters, but they just shrug. They lack the vocabulary to tell me why they simply found the game “okay.”
This is a tough case. You’ll see it with inexperienced gamers. You have to watch their body language closely as they play. Are they leaning forward, paying attention to the board? Do they complain when a player takes something they wanted? Do they ask for rules clarification? Do they take their time figuring out their move? These are all possibly signs that the player is actually engaged in the game.
On the other hand, are the players leaning back? Are they texting on their phones, or on social media? Do they make moves without really caring about implications? If so, they are not engaged in the game. It could be the player, or it could be the game.
In these examples, rely on your experience with this gamer, and with other gamers who have played your game. If this is the only player who’s detached, and you don’t know him very well, then it may just be an issue with him. But if you repeatedly see players getting detached from your game, you will want to study your game. Is its core engagement working? Should it be shorter? Should it be more tense and dramatic?
Example 2:
We finish playing. Just like before, there is silence. I ask, “What did you think?” Everyone says, “It’s okay. Not bad.”
Now, these aren’t casual gamers this time. These are playtesters I know and trust. There’s something missing in the game.
We talk. The game works well enough to be playable, and kind of fun. But we can’t figure out what would have pushed it to the next level. The game is at the dreaded good-but-not-great stage.
This is really bad. It’s one of my worst game design nightmares. If you come across this, one of the best things you can do is just put the game down. Work on something else. Maybe take a break from game design altogether; not because you’re bad at it, but because sometimes, creative silences are good. I did comedy for a year and a half, and I returned to game design refreshed and energized.
Whether you work on another game or some new enterprise, you’re gathering new perspectives. When you return to your okay game, you might find some place to insert that missing spark.
Example 3:
We finish playing. But now, everybody LOVES it. They rave about the game. They ask where they can buy it, and act stunned when I tell them it’s still a prototype. They can’t think of any way to improve the game. I feel incredible. Dr. Knizia, look out!
So we go to another game. Maybe it’s a game they’ve never played before. They LOVE this game too. They rave about it. They write down its name, and make sure to buy it when they can.
At some point later in the game day/convention, I see the group playing amongst themselves. They’re playing a game I detest. And they LOVE that game too. They talk about how great this game is, and how, if they never had to play any other game, it would be totally fun. They tell me that my game is just as good as this game.
And I realize, my heart sinking: these guys just LOVE every game they play. Maybe they were being polite to me (I love Ignacy Trzewiczek’s story about this). Maybe they genuinely have no capacity for criticism.
Honestly, this is a waste of a playtest. Your playtesters should know to be honest. If they’re not enjoying the game, they need to say so. They do you no favors when they’re polite to you. And if they’re being honest? Just toss it into the aggregate. Maybe, just maybe, you actually have a loveable game. (Note: I like you and everything, Dear Reader, but chances are, your game isn’t that loveable yet.)
Again, reading your testers is vital. If one player just got unfairly screwed by something, ask her how she feels. If she laughs it off, make sure she understands that you need her to be honest. She should then give you the truth: either “yeah, okay, that was a bit of a raw deal. I should have a chance to…” or “no, really, it was my fault putting myself in that situation.”
In Ignacy’s words, don’t trust your playtesters. Make sure you get the truth from them, even if you have to dig.
Example 4:
You play with That Guy.
You know how it is. Maybe he’s starved for attention. Maybe she was never properly socialized, and someone is bringing her to the group as a “project.” Maybe he’s somebody’s Significant Other, and doesn’t really want to play. Maybe she’s genuinely sweet and nice, but just has no clue how to play a game competitively.
In any event, the session is ruined. This player plays in a way that makes it all about herself, and in order to get any signal, you have to sift through a whole lot of noise.
It’s strange to come across competitive imbalance issues when playtesting, because most gamers understand that the important thing is improving the game, not winning. But this player doesn’t get it. He breaks everything by playing in an unreasonable way.
Should your game hold up against this? Sometimes. But as a designer, you can only make a good game with the assumption that everyone else is trying to make logical, competitive decisions. It shouldn’t break if someone makes a poor decision in an attempt to be logical and competitive. But if someone is deliberately making illogical, game-spoiling decisions? That’s not something you can necessarily address out of the box.
Ask the testers if they’re enjoying the game. Don’t be afraid to cut it short. And remember that if you have any say in the attendance of this event, you may want to leave That Guy out of it next time.
Those four examples are all Bad Playtests.
But a playtest where your game breaks? Whether it’s truly a bad playtest is up to you. It could be the most important test of your game’s development.
Formal Ferret Games 
Good thoughts, thanks. One other “bad playtest” experience I had: I verbally explained the rules but left out two key things, then went to another table to play with some others. The first group hated it, found it dull, etc. Took a ton of prying to figure out what I’d done wrong.
I’m not game designer myself, but I’ve done plenty of play-testing. I’ve also done some writing and gotten similar feedback from my beta readers. The one time a guy trashed my story was also the best bit of feedback I got on it, and it made the whole thing better. It was also the most difficult to hear.
On the flip side of the fence, having a designer not listen to my playtest feedback is equally frustrating — I just spent 3 hours of my life playing your game, the least you can do is listening to what I have to say. I know how to give constructive feedback, I know how to describe what I see as a problem and then (maybe) suggest solutions. I don’t just say “you should change this” without being able to articulate why (usually; there is a rare occasion, as you say, where no one can quite put their finger on what’s not right).
I don’t care if you agree with my feedback or not, but don’t tell me my experience was wrong. It was my experience, end of story. If you didn’t intend for it to happen, figure out if it was an abberration or a problem with the game itself. Or even determine that it was exactly what you were looking for and I just didn’t like it, which is OK; a game doesn’t have to appeal to everyone. Just don’t tell me I’m wrong.
I understand your frustation. As a designer myself, I try to be polite about all feedback and I write every suggestion down, including the ones I know in the back of my head I’ll never use. Honestly, sometimes you just can’t tell what is good advice and what isn’t until later on.
Sometimes you make a change, and then an idea that seemed whacky before suddenly makes sense in that new dynamic. Or, perhaps after stepping away for a while you are able to see things from that person’s point of view in a way you couldn’t before.
I have noticed another somewhat related problem though; sometimes people ONLY like a certain type of game (say: train games), and they make generic complaints like, “it just wasn’t fun,” because they don’t like the kind of game and not because there was something wrong with THAT particular game.
I don’t know how to fix those problems, but I agree that designers should be grateful for all feedback, and I agree with Gil 100% that all playtests teach you something. Every person who plays our games is a potential customer in the wild, and their experiences are valuable, even if their experience is: meh.
There are some useful points here. Yet after reading your title, I asked myself, “have I ever had a bad playtest?” I usually get the bad, as in “this game is a mess”, out of the way when playing solo. The “game is just OK” playtest isn’t bad in itself, as one sample is too small to make a judgment. Also, as long as you gain useful information about the game, how is that a bad playtest? The playtest isn’t the problem, it’s the game.
I especially disagree with Example 4. If one person can break your game, doesn’t that mean your game is flawed? And needs to be fixed? Yes, I suppose random or deliberately bad choices by a player aren’t desirable, but you still need to know how your game can handle that. If it can’t, it needs to be fixed. Seriously, there are many, many players for whom winning (or successful steps toward winning) is not the major objective in playing the game. I know players whose main aim is to cause mayhem. That game must be able to deal with that, because it’s going to happen. Your discussion comes perilously close to that fallacy of design, “that’s not the way the game is supposed to be played”. If it’s within the rules, someone will play it that way, and your game has to cope with that.
Regarding Example 4, it’s one thing if a person breaks your game while being competitive. That is ABSOLUTELY something to fix. But in that example, I’m pointing out a person who breaks your game while making non-competitive moves. That player is is just being a spoilsport.
I’m not talking about incorrect decisions while trying to be competitive, I’m talking about deliberately suboptimal moves to achieve something outside the game. For example, repeatedly attacking the player in last place because he doesn’t like her. Or laying tiles in a particular pattern because they look pretty, without any regard to how they actually play. Or throwing a game deliberately to another player to win his/her affection. I remember seeing the second example in a game of Railroad Tycoon, and the third in a game of El Grande. Those aren’t half-baked prototypes; they’re extremely well-regarded games.
It’s nice to have a game that stays robust in these cases, but I don’t think a game designer is obligated to nerf all the ways a player can deliberately, uncompetitively grief other players, or exclusively help one other player win. Like I said in my Competitive Imbalance post, these issues are more representative of game group issues than problems with a particular game.
Again, all of this only applies to “spoilsports”, players who deliberately make suboptimal moves to achieve a goal outside the game. If a player makes a competitive move that ruins the game, then it’s a problem that the game’s designer must fix.
I tend to agree that the ambiguous playtest (your Example 2) is the worst kind of result, if only because it leaves you in a state of limbo — you have an idea that genuinely seems interesting, but there’s nothing obvious to add or subtract that will make it actually work. But I think we tend to mislead ourselves into believing that there necessarily is such an addition or subtraction, if only we look hard enough to identify it. The reality is that some games are just destined for ok-ness, and there’s no amount of effort that is going to make them great, or maybe even good.
I think that playtesters can see more easily than designers when a game is on a track for mere good-ness, and this is usually what leads to extreme or outlandish suggestions. As Joe notes, designers sometimes tend to bristle at these suggestions, and to dismiss them — “your suggestions would make the game different than the one I’m trying to design”. Most playtesters are too polite to say in response, “yes, but the game you’re designing isn’t going to be anything special!”
It should be noted that there is a place in the world for games that are “just ok” or “perfectly pleasant” or “harmless”. And there’s nothing wrong with investing a lot of time and energy to perfect such a game, if that’s what you want to do. But as you say, sometimes it’s just better to work on something else, if you have lots of other ideas waiting in the wings.
Great article Gil. As you mentioned, body language and being a good judge of player’s reactions is very important. (Knowing your play tester’s likes, dislike and quirks is important too) Being able to “sense the vibe” and then separate any negative experience that is created by the game from that created by outside forces is key. I think it’s an art form.
Also making good decisions about what is good feedback, and what is just an individual’s personal preference is key.
I had a session recently that was similar to example 2 although I did get good feedback. The week long depression after eating my humble pie was horrible. I thought I was so much closer to a good design. Now, a month later, I have just about finished my almost total redesign with a much better game. It was a very good lesson, but a little hard to swallow. The better I get a swallowing those days, the sooner I will make my “holy grail” (dream game).
[…] What is a bad playtest? – Fail Better […]
[…] contributions to board game design come from my outside interests.” – Kevin Wilson “A playtest where your game breaks? Whether it’s truly a bad playtest is up to you. It c… – Gil […]