Do you have a list of bullet points located anywhere on your site?
Or perhaps a list of product or service features?
Better yet, a prominent list of benefits that help sell your wares?
When did you last work at improving your list of bullets – at refining it for maximum readability and persuasiveness?
A while back? Never?
I’m here to recommend spending some time working on your bullets… whether they’re a list of benefits… of reasons to believe… of key differentiators. No matter. It’s worth it if you want people to read them and take action.
(Psst… despite what copy and CRO gurus say… just because you create a list of bullets doesn’t mean people will read ‘em – scannable or not.)
The Bullet List A/B/n Test
The awesome team at Precision Nutrition (PN) wanted to test one of their more prominent bullet lists to see how it might impact the number of people who joined their Nutrition Certification Program presale list. (Over the course of the last 3 months, I’ve worked with PN to optimize many of their key landing pages. They’ve got a great testing culture.)
This particular test was on a page that gives interested fitness pros the ability to get early notification when the certification program opens for registration (there are 2 intakes per year):
We’d experimented with those big blue ‘reasons to believe’, as well as the sign-up form, and managed to produce some decent gains.
Two elements that we hadn’t yet touched were (1) the headline and (2) the bullets mid-way down the page.
After reviewing the page analytics and our timeline, we decided to run a 6-way split test, with the following creative:
Default landing page
VARIATION 1: Default bullets + new headline #1
VARIATION 2: Default bullets + new headline #2
VARIATION 3: Optimized bullets + default headline
VARIATION 4: Optimized bullets + new headline #1
VARIATION 5: Optimized bullets + new headline #2
The 6-way test enabled us to reliably isolate the headline changes from the bullet changes. You could set up a multivariate test to do the same, but I usually gravitate to A/B/C/D…/n tests due to their simplicity.
To be clear, the majority of the page stayed the same; we preserved the logos, the 3-column reasons to believe, and the form. The headline and bullets changed. Here are the alternate variations:
The 2 new headlines were focused on program benefits. Our hypothesis was that a benefit-driven headline would persuade more visitors to submit their email address.
The remaining treatments incorporated headlines from the default page and first 2 variations, and also used a single column, copy-optimized list of replacement bullets:
What’s different about the bullets?
- We reduced the number of bullets from 9 to 6
- We began each bullet with a graphical checkmark
- We focused the strongest benefits on the first and last bullets
- We improved the sight-line by going from 3 columns to 1
- We made the copy for each bullet more concise
So what’s YOUR personal take on how we treated the bullets? What else might you have tried?
The Results of the Bullet List + Headline Test
This test was set-up in Optimizely and exposed to 100% of visitors, with equal traffic distribution across the 6 pages. The single success metric used for this test was email address submissions.
About a week after launching, we saw 2 under-performing variations and “throttled down” traffic to them. Shortly thereafter, we paused those same 2 variations in Optimizely to allow more traffic to be exposed to the high-performing creative.
And at the 9-day mark, we throttled down traffic to the default page to allow even more visitors to see (and convert on!) the winning recipes.
The test had been running for 12 days when I grabbed the screenshot of the results:
What’d we learn?
- The bullet list improvements had a massive effect on visitor behavior. A 25% increase in opt-ins means thousands of additional email marketing opportunities per year for Precision Nutrition.
- This visitor segment (fitness pros looking to be certified) is already a motivated bunch, but the bullets amplified their already-high motivation.
- It was a “clean” set of results. You’ll notice that the 3 winning recipes all include the new layout for the bullets.
- The headlines appear to have little impact on these visitors, with the conversion rate range from lowest-performing winner to highest-performing winner at just 3.3%. Time to test more headlines!
Would you make any additional observations about the results?
Leave them in the comments below…
Finally, there are some things to be learned from how the Precision Nutrition team looks at testing:
- They understand the tremendous business value of good copy!
- They are willing to test messaging, layouts, and page functionality that sometimes makes them uncomfortable
- They always want to iterate on tests… winners and losers
- They always roll out the winning creative to maximize their conversion rate
- They make creative resources available to jump on new ideas and experiments
- They are curious, customer-focused, very smart, and incredibly nice!
Those are 6 (well, technically 9) attributes I recommend you aspire to in your own business, particularly items 1 – 4 for your conversion optimization program.
Until next time!