Using Reviews to Improve Games

I don't like reviews. I've been a professional video game developer for 9 years now and I've released quite a few titles. Loading up Metacritic.com or a review site is always nerve wracking, to the point where I have just stopped looking at them. BGG's community is no different. If you've published a game, you know that scrolling past that first page is just asking for pain.

But, reviews are quite valuable. They provide a quick aggregate view of your customers' feedback. Not only that, but reviews are often coming from your most vocal and enthusiastic (for better or worse) customers. You know, those who care enough to go onto a website to review your game. This sub-group is very important as they will be your evangelists and detractors.

For perspective, just a bit over 10% of the people who own Farmageddon have noted that they own it on BGG. A tiny fraction have commented.

Today, I want to discuss simple ways to take advantage of your reviews to make better games and have a better relationship with your customers. As Farmageddon is my only released title, I'll use it as my primary example. It's not quite appropriate for me to use The Sims from work for this forum.

The Extremes Serve Nobody: I believe fairly strongly, with no data to back this up, that the extreme reviews aren't terribly useful. By this, I mean the 10s, 1s, 2s, and 3s.

A 10 should mean the game is perfect and could be played for years and is just outstanding. Also, it means you, the reviewer, just love it. A 10 holds great personal appeal. Some of us try to relegate our 10s sparingly, others do not. And while those games DO exist, it's difficult to really take advantage of such feedback. A 10 is a deeply personal reflection of something the reviewer loves. Understanding it requires you be them.

A 1-3 should mean the game is utterly broken, does not work, and is just a shameful creation. While these games exist, most of the time, a game isn't that bad. I find 1-3s are often a backlash against a particular mechanic, play style, creator, or pet peeve.

For Farmageddon reviews below about 4, you'll see the same complaints over and over: Purely random. Purely luck. No strategy. Take that. Waste of time. The 1-3s aren't people who love take that filler card games. It's not that Farmageddon is the worst of its kind (for them) and they love Gubs or other such games. They don't like this type of game.

A 1-3 is a deeply personal reflection of something the reviewer hates. Understanding it requires you be them.

You'll notice I repeated myself. You cannot rely on those who just get it to represent most or even many of your customers. Nor should you try to chase people who just fundamentally don't appreciate your offering. Farmageddon will never be the game that a reviewer who gave it a 2 will appreciate.

The extremes serve nobody.

Pluck Low Hanging Fruit: Ignore all numbers, not just the extremes, and instead catalog the qualitative complaints against your game. You cannot action against whether your game is a 5.7 or an 8. That's just not quality input.

Instead, scroll through the comments for reviews between the 4 and 8 range. Create a spreadsheet and group the comments by type. You'll often find a few consistent notes.

For Farmageddon, the game's recurring thorns are:

  • Can Mirror Bean be destroyed with a Flame Fruit?
  • Can I steal a Crop using Genetic Super Worm?
  • Can I Foul Manure a Foul Manure?
  • And a few others...

These are clear and easy opportunities to improve your relationship with your customers in a few ways:

  • Create and update an FAQ.
  • Respond to forum threads with clarification.
  • Write blog posts and designer diaries explaining your decisions.
  • Create How to Play videos that maximize focus on these key areas.

All of these demonstrate your commitment to the product, are easy methods of customer support, and will increase the enjoyment of the play experience for your customers. After all, if someone is playing incorrectly, 9 times out of 10 that means the game is less fun. Unless, of course, you didn't test your game sufficiently to determine that. But, we don't do that, right?

Also, if you're lucky, you can include these tweaks in future editions and printings. For Farmageddon's second printing, we made 3 tiny rule tweaks, one of which was a change in one word. It makes a big difference. Being responsive to your consumers shows humility, dedication, and is such an easy win for all parties.

Find the Holes: In addition to the easy, low-hanging concerns for people learning the game, you'll also find holes or criticisms of the design itself. You'll find opportunities for expansions to address concerns, or you'll learn for the sake of future games. For example, with Farmageddon, I found a few issues that I wanted to address with the expansion, Livestocked and Loaded:

  • People wanted a little more strategy amid the volatility. Farmageddon will never be Agricola, but adding in Livestock as a long-term strategy really broadens the game in a great way.
  • There needed to be more uses and decisions around low-level crops. Now, you can discard planted Sassy Wheats to Feed animals.
  • Some people felt frustrated by lack of control of Action cards. Some of that comes with the game, but the Farmer's Market Action, as well as the Livestock Actions, give players more choice over their path.
  • With Livestock and Loaded, the 2 player experience is far richer and more compelling. It becomes less a slug fest and a little more cat and mouse.

I've been very fortunate that Farmageddon has sold well enough to allow me to improve the overall game and address the critiques of my fans. Now, let's say it hadn't sold well and therefore no expansion would be forthcoming. It's still useful to know the critiques so I can address them with future games. Some ways to see this in my current designs include:

  • Putting more thought into iconography and graphic design sooner to facilitate learning.
  • Creating a glossary up front for a game so that cards use fewer words that are more consistent.
  • A better understanding of balance sooner.
  • A better understanding of broader strategies.
  • A better understanding of luck, interaction, and variance.

Your critiques are a gift, especially those in the middle range. I believe, again, without data, that those in the middle range have played your game, understand it, and are providing a more rational critique. Those at the extreme ends of the spectrum are on a tilt, either an extreme high or low, and are less likely to provide you actionable and honest input.

Thanks for the Review: This is a parting note, and a suggestion from years of observation and experience. If you receive a negative review, and you will, you're allowed to do one thing: Post, "Thanks for the review." You may also try, "I'm sorry you didn't enjoy the game. Thanks for the review." Not everyone will like your game, especially when you make a game with certain highly contentious elements. You have to recognize that opinion. Note, I didn't say you need to respect that opinion.

By appearing to say "Thanks for the review" you do a few things:

  • Demonstrate that you read all reviews, not just the good ones.
  • Demonstrate that you're an adult and can take the criticism. This is a VERY important skill for designers.
  • Gives you an opening for dialog. When they see points 1 and 2, they might engage with you further. Making a friend now will pay off in the future.

As a personal example, I sent Josh Edwards of Board Game Reviews by Josh an EARLY prototype copy of Farmageddon. This is back when it was on The Game Crafter. He really didn't like it and he gave me a lot of input. I responded to it and used it to develop the game further. A year or so later when he reviewed the final game, he did so far more favorably. Yes, I made a better game. But, I also did the work to be a reasonable person.

Finally, upon reading the review, you may find the reviewer made some mistakes in reading your rules. This is an opportunity!

"Thanks for the review. I'm sorry you didn't like the game. One thing I wanted to note was that you made one slight error in regards to a rule. I'll make sure I update the FAQ so others don't miss it! Thanks for pointing that out. Instead of doing X, you want to do Y. Hope that helps."

It is unlikely, honestly, that you'll win that person over. But, others will see this dialog, will learn from it, and will appreciate you being a reasonable person.

I hope this was useful for you. What advice do YOU have for taking advantage of reviews?

Comments

Have you only ever considered reviews that leave a numerical score? I write reviews and never leave a score at the end and so far I've stood by my guns that the number at the end is far more subjective than then positives and negatives outlined in the review itself. Is this something

I should be doing in future to help differentiate games for the designer? Is it something people prefer to read?

My vote: keep the number out.

I loved this article. i agree with keeping numbers out of the system.