Post by: Grant Rodiek
I believe testing is the most important aspect of game design. Perhaps it's because I lack the brilliance of Feld or Don X., so I require testing to improve my ideas. Ah, if only I were brilliant. Nonetheless, I fundamentally believe that an idea is just that until it's proven with actual players.
Many people don't test enough. In my past, I haven't tested enough. However, testing is one of the few things within our control throughout this process.
We cannot control the opinions and tastes of our players. We cannot dictate the whims of consumers and the environment in which people play our game. But, through testing we can ensure we have perfect rules, streamlined mechanics, and focused games for our players. Here are some of the things I've learned through testing. Note that some comments pertain to iteration, but I often link testing and iteration as one.
Know your game and your goals for it. Do not implement a tester's feedback because they gave it. I don't know the exact quote, but Matt Leacock said that many games meander for years because they lack focus. Don't meander! Implement and adhere to suggestions because they are right for your game.
Every tester has a different favorite mechanic ("I love drafting. Have you thought of that!"), favorite way of doing things, or something they hate in games ("Ugh, luck? I hate luck."). Keep this in mind. If you chase after every tester's favorite thing, your game will become a hodge podge of mismatched mechanics. Use their input to make your vision better and more fun.
Here's a favorite tactic of mine: When a tester says "I didn't like this mechanic," instead of telling them they are wrong, I explain to them why it exists. My exact phrasing is often "Here's the reason I did that. Perhaps you can help me improve it?"
For example, in a recent test of Field Marshals I was told the math for calculating the results of a battle was clunky (truth). They suggested something that was indeed simpler, but it did away with the attrition model (based on the Franco-Prussian War, among others) I sought to recreate. I explained my goals and through this we arrived at an implementation that was far simpler and met my goals for the feature.
If you're curious, here are my guiding goals for Field Marshals:
- Create a highly accessible war game ( i.e not much more difficult than Memoir '44)
- Plays in an hour or less
- Plays with 2 or more players (many war games are just 2 players)
- Use a card driven mechanic (instead of dice)
- Leverage the tactics and weaponry of the 19th century
Know the difference between a goal of yours and a bad feature. This comment is meant to balance the previous one. If you let yourself, you can counter every tester suggestion with "that conflicts with my vision." Don't sell yourself on the superiority of your own ideas. Listen to your testers, ask questions to understand their perspective and comments, and evaluate it as openly and reasonably as possible. Use the data, don't thrash against it.
Do not insult your testers or let them feel insulted. Nobody wants to test for a jerk. Don't get angry when you receive harsh feedback. Instead, push yourself to make the game much better. People will only tell you they love they game if they fear you won't accept their criticism. Make sure they feel comfortable and appreciated for telling you what you really need to hear. Trust me, you need to hear it. As a side note, I feel incredibly blessed to be surrounded by an awesome group of testers. I've been a professional game developer for 7 years, so I have a fantastic group of about 10 people who are incredibly enthusiastic about games, are well-versed in design, and aren't afraid to be honest with me.
Enter your tests with a purpose. Try not to change everything before a new test. Obviously, game design isn't science, but the closer we can move towards a "control" environment with a few variables, the better our data will be. If you change everything, it'll be really difficult to know what worked, what didn't, and more importantly, why for either.
If possible, have a tester read your rules, then explain them to the group. This more closely emulates the real world playing of your game. Most people learn to play a game after one person reads the rules. Unfortunately, you don't ship with each copy of your game that's sold. If you cannot find a tester to read the rules beforehand, try to explain the game in the same order of your rules. This will help you get a feel for how to better organize the rules. Plus, when people ask questions like "How do I use that thing you just mentioned," you'll get the tip to bring that information forward in the rules.
Test with a wide variety of people, including people you think will love your game, hate your game, and somewhere in between. One of the most interesting tests I've held for Field Marshals so far was with two incredibly competitive gamers. That's not the type of gamer I am, nor is it really my focus when developing games. But, observing how they viewed the game and broke down every element to maximize their chances for victory was fascinating. Had I only tested with casual, lightweight gamers (like me), I would have missed out on this perspectives.
If someone asks a question, take a note of it. Questions and confusion are a great indication that a feature was poorly explained or may need to be streamlined. Before I put Poor Abby on hold, I conducted a test with my new shiny prototype. The tester asked questions on the Argument cards for a solid 5 minutes. He just did not get them! My takeaway wasn't that he was dense. Instead, I realized that if this veteran, hardcore board gamer couldn't grasp the mechanic, perhaps it was due for a complete reconsideration.
It's not you, it's me.
Take notes on what features/mechanics/player actions are used and which ones are ignored. If a feature is ignored by the majority of your testers, it is either too clunky or doesn't clearly provide enough value. I've found that 9 times out of 10, testers will take a less ideal move using a mechanic that's more clear. If it's confusing, they'll just ignore it. Regarding the value comment, Sid Meier often notes that when balancing a game, either double the value, or cut it in half. If nobody is using a feature or action, you may need to double its value. This tip has been ridiculously useful in refining the cards in Poor Abby and picking which Tactics to list for Field Marshals.
Review your notes with the testers at the end of a test. I always like to quickly read over all the things I observed and changes I intend to make. This serves a few purposes. One, it demonstrates to them that you intend to make changes based on their feedback and didn't just waste their time. It also gives them a second chance to ponder and think about things you may have been discussing the entire time. You shouldn't end a test without discussing the mechanics, the flow of the game, and everyone's opinion. This is a good way to start that conversation.
Blind Test. In case you aren't familiar with the term, a blind test is when you send your game to someone who isn't familiar with the game, has to read the rules to learn to play on their own, and ideally, doesn't even know you. A blind test is as close to a live fire exercise. I found blind testing on Farmageddon to not only be invaluable for improving my rules, but often, when you have a stranger play your game, they'll be frank with you. I typically found my blind testers by checking to see who was interested in Farmageddon. People would mark the game as "Want to Play" or "Want in Trade" on BGG. I would send them a message and if they were interested, a copy of the game.
Be willing to kill your favorite features. When you enter a test, do so with the resolution that nothing is sacred. Wait, that's wrong. What's sacred is that your game should be fun and that nothing should hinder that. I had this nifty random turn order mechanic for Field Marshals that I kept trying to tweak, streamline, and improve. Unfortunately, after a dozen or so tests it was clear that, while the rest of the game was really improving, the turn order mechanic wasn't the sharpest tack. I finally threw it away and re-examined how to determine player turn order. Guess what? The new mechanic actually adds depth and strategy to the game. Shocking!
Test Hunches. You want to be careful about fiddling just to fiddle. The majority of the time, you want to make changes based on problems you witness in tests. You want to fix issues, not create new ones. However. While you're testing, if you have a hunch or think something might improve the game, try it. For example, I was curious whether changing a player's limit of "play 4 cards per turn" to "play 3 cards" would improve the game. In reality, it made the game significantly worse. Now I know! I also better understand why it made sense to not just revert to 4 cards, but modify a few things and make it 5 cards (with other changes). My bad idea led to the right idea.
What have I missed? Care to add anything?