Thoughts from Hocus Blind Testing
Post by: Grant Rodiek
I wish to thank Mathew Tate for submitting the photo above. His hands are lovely.
Around November or early December, Josh and I sent approximately 10 copies of Hocus Poker to testers around the US (and one in Denmark!) who agreed to play the game for us. We paid to have the cards printed via DTC, as it's about $6 per set (plus a few more bucks for shipping). That seemed worth it to us, as we didn't have to cut out 10 sets of cards, and neither would our testers. If you have a small game, we think it's been a resounding success. Especially if you don't have a big name reputation like Plaid Hat Games or Portal, it's a great way to get involved blind testers.
The results have been very good. Around the time of BGG we thought our game was actually good. We knew it had spell balance tweaks, and that the rules needed iteration, but we thought the version we had was the final one after some development. That's a bold proclamation. Thankfully, the feedback has been very good. We had one tester player 13 times in one weekend, another played 8 times over a few weeks, another has his friends asking to play it when they come over. That's good stuff.
Often times when people discuss development, it's about a game that's still broken, far from finished, or deeply rough. That's most of what you read about on my blog, for example. And, if you click on the Hocus Poker tag, that's the main body of content. However, because Hocus is mature and very far along, I thought it might be interesting to discuss some of the big changes we've made, as well as some of the interesting thoughts that have occurred to us.
Tweak the rules, get frustrated, throw them away, start over. I wrote about this process extensively here, but it bears repeating in this post. We made, probably, a thousand tiny changes to our rules document. That was expected. What wasn't expected was that our rules would become this lumbering beast, more confusing than before. Don't believe us?
What do you think?
The urge for 5 player becomes strong. Josh and I have always wanted 5 player, but a while ago we said "it's too hard" and set it aside. True, we had some limiting factors: the size of the deck (52 cards) affected the ability to draw cards, and 52 cards is standard for poker. Nobody really mentioned it, but I think the fact that a.) things were going well and b.) it's such a massive win for the product made us antsy.
I don't think we would have arrived at this on our own from local testing. But, getting the confirmation from so many that things were going well freed our minds for boldness. Game designers NEED confirmation checks ins from others. We NEED validation. Without it, we'll crumble. Or, you're probably a little too arrogant?
Our 5 player solution is simple, though to be clear it still needs testing.
- Added a 0 and 14 Strength card to every suit. This makes it a 60 card deck, but by and large keeps the Poker-ness whole. These cards are only added for 5 players. This solves the draw problem and keeps the distribution of hands with 5 players more sane.
- Players are dealt 9, instead of 10, cards. With more out on the board, and the need for cards to draw, this again reigns things in.
- There is a third Community, with its own pot, but players can still only have 2 pockets. This mean that we don't have 5 players making a mad stupid rush for 2 communities. But, it also adds a neat layer to the strategy: which 2 communities are you vying to win? Which one are you skipping? Perhaps more importantly, which community are your opponents skipping?
With external validation, your mind will be freed to solve good problems once again. Seek out legitimate validation.
There were subtle trends we weren't noticing. Last night, the wonderful Marguerite Cottrell mailed us a personal VIDEO of her notes with Hocus Poker. She'd played 8 times, with and without Advanced Spells. She succinctly offered high level notes, thoughts, and gut reactions. Then, she went through each individual Spell Book (like Alchemy, Illusion), and gave her personal thoughts on it, identified its weakness, or its imbalance.
In a few of these, she revealed two enormous Gems. Josh and I hadn't thought of it this way prior and when we heard Marguerite say it, we simultaneously thought "Oh, yeah!"
- Spell Books (a set of 3 Spells a player receives that are unique to them) that have a Spell that does "Do a unique thing. You may then do a Basic Spell." were more powerful than Spells without. She's right. It's essentially a Spell that does 2 things. We're spreading the love, now.
- All Spell books tend to affect 2 of the 3 areas of the board. By this, she means Community, Pots, and Pockets. She noted some of the weaker Spell Books only affected a single thing. Great insight! Again, we're spreading the love.
Find someone like Marguerite (stay away from her we need to send her more projects) to cut to the heart of an issue. If you've been a designer for a while, you know how frustrating it can be as a designer to be bombarded by tester requests for stuff or changes without reason. "You can add this. You should change this. Why isn't this like this." As Gil Hova noted to me at BGG in a discussion, "Please tell me the problem, I'll find the solution."
Battle lines have been drawn around Basic and Advanced. Basic mode is, in my opinion, a wonderful addition to the Hocus Poker product that is entirely an accident. The game I took to BGG on me and Josh's behalf was 3 Basic Spells, 3 Advanced Spells per player, and Moonbears. A publisher, after my pitch, noted the game was too complicated for what he wanted and asked if there was a simpler version without the Advanced Spells.
My answer was "I dunno! That's a fascinating idea." I wrangled some friends at the con, then my family again at Thanksgiving the following week, and wouldn't you know, it was a great idea. We saw this as the tutorial version. It removed some complexity and options and was much faster to teach.
Here's the thing: some people love it, to the point they don't even want to try Advanced, or once they do, they want to go back. Marguerite's roommate noted that "Basic is more strategic with more control, whereas Advanced is more tactical." That's fascinating.
I've seen this trend in person. Some people play Basic and say "that's cool, but it needs a little more." Boom, here's Advanced. I showed my local gamer group, guys and gals who play meaty stuff, and they thought "woah, Basic is packed with decisions. It doesn't need more."
This is all entirely unexpected for me and Josh. We've made the decision to present the game as advanced (without the label), putting Basic at the bottom of the rules as an alternative. Through testing, we think that is the best way to manage expectations and put our best foot forward. But even Josh and I are a little divided. Me? I sorta prefer basic. It has a classic card game soul and I just dig it. Josh? He's an Advanced guy. We both like both, but choices are being made. Entirely unexpected outcome from a part of the game that was entirely unexpected.
If you ask me, having both is a great addition to the product. It suits different moods, different personalities, and different groups.
The force is strong with this one. Since we began these tests, we haven't changed a single core rule or mechanism (excluding the addition of 5 player). We have re-written rules, we have clarified options, we have tweaked Moonbear content, and we have thrown away, re-designed, re-worded, or simply balanced the Advanced Spells. But the core remains.
I've been approximately tracking tests from us and our testers and we're around 50 tests on just this version. That's very strong validation. We're kicking the tires and they are like "come on, man, we're good!"
Yeah, that's right. Our tires talk. We have Advanced Spells to refine and need to run tests against our final graphic design when it's ready. But, it feels so good!
In closing, a hilarious Hocus Poker story. It seems our thematic integration is a LITTLE TOO STRONG (har har). This morning, tester Robin Lees mailed me this picture. Apparently, his printed, without a command, and no computers on in the house, decided to print our rules.
It's a sign!
You can read the rules for Hocus Poker here. We're revising the Spells now, so I don't want you to waste your time printing. We'll link the PNP soon. In the future, we'll be discussing art production and other publisher related things for the game. If you have questions, mention them below. Stay tuned!