Solstice Beta Wave 1 Results
The first testing wave of Solstice has been going for a month now, and a lot of small changes are going to be incorporated into the design to make it stronger. The overall feedback has been good so far, and the testers have been great. We have a pretty active Slack channel to discuss things. but testers haven't limited themselves to that. One tester recorded a 25 minute video of her group discussing the game and answering my question guide. Another tester took a break from work to discuss the game's theme and other topics for an hour. Super cool.
I should also note, for people curious, that my experiment in selling a POD version of the game, at cost, has been successful. Several people took me up on the offer and they've generally been very engaged testers. Woo!
In this post, I want to write about the changes being introduced into the game for wave 2. I want to explain why I'm making the changes, and throughout, offer advice and insight that can benefit you when conducting a blind test program for your own designs.
Balance has performed well so far, which is good. The game is lightly asymmetric, but unlike Hocus, the asymmetry can be balanced more mathematically and is less of a feel exercise. However, there are a few small notes that needed to be addressed.
Siege is an Event that exists to hinder players who dog pile on a single region and hinder the leader. Previously, it had the following effects:
"Monarchs cannot score this region. Strength Victor loses two points."
This can be a real double whammy. If your Monarch (no Aristocrat, more on that soon) doesn't score, that's a 2-5 point swing. And if you then lose another two points...damn Daniel. The card has now been simplified and nerfed to be:
"Monarchs cannot Score this Region."
This is a pretty good and clean stopper and doesn't feel so punitive.
Regarding the player cards, they're in pretty good shape, but I took some feedback and used it to investigate some issues. Basically, players felt that some clans have much easier ways to score their 3 Point cards. Now, this is true, but if you look into it, it is a little more nuanced.
The Warchief and Vizier that Score 3 are easier to resolve than the Assassin and Monarch. However, their 3 Point Score is minimized by the fact that the other factions score 2 points. So, it's an advantage, but not a huge one. Conversely, the other Monarchs and Assassins don't score nearly as much as the others. However, in one case, a faction DIDN'T score the two points others were scoring, so I brought them in par.
I then looked at the Elders and found a few more problems. The Sea Clan could score 4, when the others could only Score 3. I brought them to par. I also noticed some of the other clans were given more points in the stats for which they weren't strong, which is an unfair tweak. That was an easy change.
Overall, balance won't see a swinging shift, but it will be brought more in line, which is key.
I finally admitted some cards weren't working and altered them. Supply Caravan has been a problematic card for a bit. It was too hard to execute, almost always resolved the same way, and didn't make the game more interesting. Lame!
I replaced it with Escape.
"<Favor> Victory: You may add your Prisoner to this Region (ignoring card limit). It resolves normally."
This is the first card to leverage prisoners, which makes it interesting. If you have this card, it can/will change how you use the prisoner, and can lead to a very surprising result.
Although not a direct result of the testing, we're also rolling out the B Sides of the Regions. This is something we've been discussing behind the scenes, and many testers echoed a desire for such a feature. Essentially, there are the plain A Sides to every region, which just reward points. The B-Sides, however, reward fewer points, but grant players bonuses to resolve. This will change the game and add a new strategic layer without too much complexity.
The final content change is that we added a new disclosure rule to add variety. On the coast, you now disclose the card's strength or favor values.This adds more variety.
There haven't been dramatic changes to the rules, but there is one that I think will really improve the game.
Players are now dealt a random card that is a prisoner at the start of every game. This has two subtle impacts. One, it increases the number of cards in play, which further reduces the already unlikely chance one player has none of their cards in play. Secondly, it removes the exception that players do not have/cannot use prisoners in round one. Now, all rounds have all content.
Otherwise, there's a minor rule change regarding Region use. It was noted that players felt the need to control THEIR region, but there are no rules for that. But, previously, regions were associated with different clans so to aid in setup. For example, if you're playing with the blue and green clans, you simply toss in the blue and green regions. No more. I removed the clan affiliation from the Regions. Now, you choose regions at random equal to the number of players.
To make the game more accessible, I made a few tiny changes that I think will have strong implications. Firstly, I re-positioned some of the card diagrams to the top of the rule. This way, you have them in mind while learning the rules.
I designed a set of quick start rules for first time players. Effectively, players are dealt a specific set of cards, and use a preset pool of Events. This lets them have simple Events for their first game and they skip the drafting phase. Many players are scared or uncomfortable drafting cards before they know how to use them. This alleviates that.
I separated the two player rules for quick access.
I put all Prisoner rules in a single section. I reference them elsewhere, but I put them in a single place so you can learn them all at once.
Some players were missing the "Play face up" text on some cards. I made sure to bold and underline this text. I did a similar thing for "Discard a Farmer card" on a few cards in Farmageddon and it did the trick. Humans are really bad at glossing over information. Help their brains out and add subtle call outs to key exceptions.
I tweaked elements of the overview and added a few snippets of high level, guiding text to help frame the game for players. For example, I note that favor tends to grant powerful Event bonuses, whereas Strength tends to reward points. The initial overview does a full step by step of the 3 key decision points in the game, instead of glossing over it. These are subtle changes that only strengthen a player's ability to learn the game.
I added a high level description of a clan's strength to the back of the reference card. For example, it'll tell you that the Mercenaries are dominant with their military, and have a total of 8 strength and 5 favor. At a glance, you now know what you're good at.
I changed the X on some cards to a 0. The X was meant as: this doesn't resolve in order, it just is. But, the X was misleading. In one case, a tester noted that it reminded them of Magic The Gathering, where the X means a conditional variable. Always remember how other games use language! By making it a 0, players read the cards first, so they can resolve them before any other. This is such a "no duh" change that I'm ashamed I didn't think of it sooner.
Finally, I added new diagrams to explain more situations in the rules to better show how cards resolve.
I had to conduct a bit of theme re-work to mitigate some disconnects and improve the experience. Without art to help me, it's tough right now, but it's important to work at it as much as possible.
Firstly, many of the cards had name changes. Most of these are to accommodate the final art and presentation, so I want to change the names NOW to take them for a spin.
Secondly, I re-wrote the game's introduction and premise. I wanted to better frame the conflict and the characters involved.
Thirdly, I removed the notion of clans. The final game will not be fought by geographic factions, but different ones in the same location. The game is now about the Merchants, Mercenaries, Wizards, and Seers. Four groups with different visions for the future. The players are Machiavellian figures manipulating these groups from the shadows. There was a concern, that's best highlighted with the question:
"Why the heck is MY Monarch going here? I didn't put him there!"
The idea behind Solstice is that you don't have perfect control. You aren't directing your characters. You're merely doing what you can to move some people to one place, thematically alert leaders that a Monarch is there, and should be assassinated. If you look back 300 years, conflicts were very difficult to fight because allies couldn't communicate like we can now. Hell, 100 years ago in World War 1 it was practically impossible to coordinate an assault beyond shouting distance.
The name changes are intended to support the fiction I'm positing. Some people might always have a slight...break with that, and ultimately, I have to accept that because this is the game's secret sauce. The fact that you can draft and play other player's cards is important and is one of the neat things the game does.
Shifting the factions around had a few implications. I had to move the military cards to the Mercenaries faction. It didn't make sense that they weren't the strongest in the military!
Lessons and Things to Keep in Mind while Testing
Blind testing Solstice is eerily familiar to Hocus, Cry Havoc, and Farmageddon. There are things that are always true, which, if you know, you can leverage to conduct better testing.
Testers are good at finding problems. They're not always good at finding solutions. When testers share a frustration or a dislike, don't ask how they would fix it. Ask why they don't like it. Ask what they want to get out of it. Ask what experience they want to feel. Use that information, and knowledge of your design, to address the root cause. I had one tester recently note I should make Solstice a deckbuilding game to add more player control. Aka, I should completely make a new game! Focus on the why, not the "how to fix."
Testers will sometimes say crazy things, and you need to ask questions to get to the root concern. Initially when testers said they felt there was imbalance, I disagreed. I had to ask, pry, and poke, and eventually I found out WHY they felt that. Guess what, they were right! Another tester had good concerns with the theme, and it took about 30 minutes on the phone to really understand his critique. You have to dig in most of the time. The initial comment won't tell the full story.
Take rules, layout, and text seriously. Every time I take the lazy route and don't update a diagram in the rules, or put off a change, it bites me. Testers always comment on these things. Take your testing as seriously as you can, and your testers will reward you with effort.
This is going to sound dismissive, but it's not meant that way. But, more and more I think it's very true. As long as a game is in a prototype state, people will always find things wrong with it. They just will. I bet that if Eric Lang took Blood Rage, a game that has fairly universal praise, but put it in front of people with prototype components, people would complain about it. How do you use this information? Well, know your game. Know your goals. Know where your game is at right now, and where it needs to be. At some point, the game will be finished and you'll need to flip the switch. If you did your job, your testers will agree.
On Cry Havoc, Ignacy and I were arguing about cards and text until the moment we hit print. On Farmageddon, I was worried about tiny issues until my core test team said "Dude, it's done. Seriously. It's good!" It's human nature to nitpick and critique things that are "in progress." We go into red pen mode. Know that, and use that information wisely.
Be okay telling testers they are wrong. There are times when your testers will have comments that are inaccurate. But, you need to damn well know they are wrong. I've played Solstice 70 times. Most of my testers haven't played it more than 5. Sometimes they will have a comment that is inaccurate. I need to be able to discuss this with expertise. This doesn't mean you can be dismissive, or arrogant. This is a good opportunity to ask questions and get to the root cause, or learn more about their perspective. Know your game inside and out, and know your goals, before you go hands off and ask others to dig in.
Not all testers speak game designer. This is useful for evaluating customer feedback as well in reviews. Testers often confuse things like randomness, luck, strategy, and balance. I'm going to say this on almost every one of these notes, but do not fixate on the key term used. Instead, ask a question to better understand their point. They might say "the game is too random" when they really mean "I wanted more control." They might say "the game is unbalanced" when they mean "I didn't feel I could recover from the point deficit." Don't fixate on words that hardly anybody uses consistently. Instead, have a discussion and get to the root cause!
Work to understand perspectives in order to understand feedback. I had a long chat with a tester who was describing some of the frustrations two of his friends were having with the game. Initially I thought, man, I need to fix this, but then we dug into the play styles and personalities of the players. It turns out, Solstice just may not be their ideal game. Now, as is true with most of these comments, that doesn't mean I can dismiss their notes! It does mean, though, that Solstice may never be a 10 for these guys. But, I should work to make sure it's a 6 out of 10, not a 2 out of 10.
Again, ask questions and find out what their true concerns are. In this case, they wanted more control. I made sure there's a prisoner in round one as a result as it gives more control and improves the probability of the card pool. These testers, who are more inclined towards Euros that have less direct player interaction in your decisions, were uncomfortable starting the game. It was tough for them to draft with imperfect information. Therefore, I made those quick start rules.
Solstice is a drafting game. It's an interactive game. There's not a lot of randomness, but players can and will upset your plans. Like with Hocus, and Cry Havoc, and Farmageddon, the game isn't about a perfectly executed plan, but making the most out of the resources and things you can control. To make an extreme example, Solstice isn't Caylus, but I need to improve the margins where I can to alleviate concerns.
There will be all types. Players who want more luck, more complexity, more strategy, more variety. Know your game, know your goals, and do your best to satisfy them, but don't water down your game. You can never make everyone happy. But, you can thrill the pants off your target audience.
This post is beginning to run a bit long! Hopefully this information is of some interest to you, and hopefully these tips are valuable. If you have any questions, comments, or feedback, post them in the comments!