Split Decision: How we split-tested a book cover

There are a lot of decisions to make in publishing. Some of them, like whether or not you need an author platform, are easy to make (the answer is yes, by the way). But sometimes we get stumped. We ran into this problem when one of our upcoming titles, Resolve by Hal Movius, was back from the designers with not one, but three great covers. We’ll call them Orange, White and Blue.

A book can only have one cover (unless it’s Taschen’s 1000 Record Covers), which means we would have to narrow those three choices down to one favourite. So we ran the gambit of input: we discussed the covers with the LifeTree team, pored over them with our distributor, and, of course consulted the author.

The results were almost comically split. We at LifeTree flagged Blue as the winner, but our distributor recommended Orange. The author wanted White.

We did further testing. We shared the covers on Facebook. The author’s friends and colleagues also loved white, while our own Facebook page was split between Blue and Orange.

The results showed us that we needed to dig deeper and adjust our focus with the testing. The author’s academic peer group preferred the more academic cover–but they weren’t representative of the book’s demographics: an educated audience with the desire to improve their conflict resolution. We had to find those people and get their honest, unbiased opinion of our covers.

Our deadline was coming up, and we needed to nail down a cover to send out to Amazon, Indigo, Barnes and Noble, and the rest of the book trade. We were trapped between a rock and a hardcover. We realized this wasn’t a decision we could make on our own–that’s when we decided to split-test.

What is split testing?

Split testing is also know as A/B testing (when choosing between two options) or multivariate testing (when multiple versions or elements are being tested). This is a common marketing tool that can include:

Showing different website layouts to different visitors to see which one gets more clicks.
Sending different email subjects to a portion of your mailing list to see which has a higher open rate.
Creating Google ads for two different book titles to see which one is more enticing.

In our case, we needed to split-test our book covers to decide a winner once and for all.

After looking at a few testing tools, we decided PickFu was the best. They specialize in pitting any two options against each other to a panel of real people that provide near-instantaneous feedback (relative to standard market testing) to any question you have.

We decided to test Blue with Orange, and Blue with White. The tests we set up looked like this:

You’ll notice we got to pick our own demographics. We wanted to match this up as closely to our intended audience as possible, so we asked for feedback from people that had attended at least college or grad school, and read at least one book a month. Here are the results:

Blue vs. Orange

As you can see, these two covers came very close–in fact, the results here are statistically insignificant. However, each person who votes also has to write a response, meaning we have 50 explanations for why one cover is preferred over the other. For example, a fan of Orange said, “[The maze] shows that negotiating conflict can be tricky and needs forethought to reach resolution.”

We put those answers aside, however, when we saw the results for the next test:

Blue vs. White

Being chosen by 73% of the 100 respondants, Blue was the clear winner of this test. But as we delved into the comments we found there were reasons to like White, as our respondents told us:

“White is striking, catches my attention and is to the point.”

“It looks stronger and more confident.”

“Blue is busy whereas White looks professional and helpful.”

“If I saw White on the shelf, I would definitely pick it up.”

And perhaps the most colourful response: “Blue looks like a 1990s tween magazine….”

In addition, some comments for Blue stated that White looked too unfinished to get their vote. While the results were a landslide for blue, the comments spurred us on to give White a little TLC and see if we could turn it into a true contender.

The result was a beautiful cover that is confident, eye-catching, and unmissable:

In the end, we didn’t choose a winning cover from among the three examples, but our split-tests gave us a much clearer idea of where we should go. Instead of hovering between three good covers, we came away with one great one that everyone–us, our distributor, the author, and his readers–will be excited to pick up in a bookstore.

This is just one way we’re modernizing publishing. Instead of relying on personal opinions – even well-informed ones – it’s now possible to test your ideas in the real world, with real readers. Split-testing–whether it’s for covers, titles, concepts, or more–is one way we do this. To learn more about how we’ve brought publishing into the 21st century, check out how we turned one book into a website.

If you want more publishing and writing tips, tricks, and inspiraton, subscribe to our monthly e-magazine.