LAST UPDATED MAY 11, 2020
Last week, Neil Patel wrote about the optimization of the Crazy Egg home page, which his biz partner Hiten Shah hired me to rewrite…
I’d met Hiten when we were speaking at Microconf, and I was stoked when he challenged me to beat a Control that was hugely intimidating. If you read Neil’s post, you’ve got some ideas on the test and the outcomes – and now you can see how I developed the winning copy. So you can do the same on your site…
The Page We Were Trying to Beat
We were up against a strong incumbent. Like, super-strong. Like, you’ve heard about it because it was the very page that had won in this popular and much-discussed A/B test by the uber-smart folks at Conversion Rate Experts…
Recognize the following graphic? If you’ve been in CRO or online marketing for more than 2 years, you probably do. It’s by CRE. It shows the Control they were against… and the [ultimately winning] Challenger they created:
The CRE Challenger brought in a 30% lift over the then-Control… and proved a major win for everyone who’s ever championed the power of long copy to convert…
Their Challenger became the new Control.
And it later became the one for me to ‘beat’.
Of course, the saying goes “Always Be Testing” not only because Alec Baldwin made “Always Be Closing” the catchphrase of every salesperson from 1992 onward… and not only because Bryan Eisenberg wrote the book on it… and not only because it’s always better to test than to guess… but ALSO because optimization is an ongoing process, so you should never stop testing…
That the Challenger here beat the Control here does not mean that long copy is better than short copy.
When B beats A, that does not mean B will also beat C, D, E, F, G or H. That’s why we have to keep testing and avoiding wholesale conclusions, like “Long copy beats short copy.” Because, as you’ll see, that’s not always true…
So with that in mind, we wanted to test the messaging on the Crazy Egg home page. For the purposes of this test, it was important to keep the general layout the same – rather than using, say, a dramatically different design, as was used in the CRE test.
We were trying to answer:
- Which messages are critical to conversion… and which might be ‘extra’ or non-impacting?
- Is leading with a benefit or outcome better than leading with a feature (i.e., eye-tracking)?
- Is the long-form hybrid style working here?–is the power of the long-form narrative coming through?
- Are the long-form-y elements, like the Johnson Box, adding value?
Here’s how we answered those questions and others, and helped the Crazy Egg team see that long copy isn’t the only way…
The Exact 7 Steps We Followed
(Don’t Optimize a Page Without ‘Em!)
In conversion copywriting, Research & Discovery is everything. You can’t write conversion copy if you don’t know what your stellar messages are. So we spent most of our time on discovery (and so should you!).
This is the exact process we followed to find the messages and messaging hierarchy worth A/B testing:
1. Start with a Recorded Assessment of the Page
Turn on Camtasia or whatever screen-cap software you use – there are free ones out there – and record yourself assessing the page you want to optimize. Think aloud. Take as long as you need to. Ask questions aloud; tell yourself to make note of X, Y and Z; tell yourself to set up a Qualaroo survey that asks X question. Say everything. Record it. Mp4 it.
And then send it to be transcribed; we use and recommend Rev.
2. Print Your Transcript and Grab a Highlighter
We’re paperless here at Copyhackers… except when it comes to this step. So we have to send our transcripts to Staples to be printed on good ol’ fashioned dead trees. But it’s worth it. (Sorry trees!) Once you’ve got your transcript in hand, you sit down with an old-school pack of highlighters and:
– Use yellow for great copy ideas / insights
– Use pink for great design ideas / insights
– Use green for further research ideas
How do you know if something’s great enough to be highlighted? You’ll probably have waited 1-2 days for your transcript… so your original assessment isn’t fresh in your mind. That’s a good thing. Only highlight the copy and design ideas that jump off the page at you.
Highlight all further research ideas. They are all good.
3. Use Crazy Egg
Yes, we used Crazy Egg to optimize CrazyEgg.com. It was very postmodern. I dug it. Although all 4 maps Crazy Egg creates were useful, I most wanted to see the scroll-mapping… because a long page is tough to get through, and we needed to see what points were capturing and recapturing attention. The whole scrollmap is huge, so here are snippets of the more meaningful parts…
SMOKIN’ HOT AT TOP OF PAGE…
THEN THE DEEPFREEZE BEGINS (AND LASTS)
AFTER EXTENDED COLD SNAP,
GETS WARM-ISH AGAIN AT BOTTOM (PRICING)
4. Do a Segmented Customer Survey
The more you can segment the survey, the better. In the case of Crazy Egg, Hiten was able to segment his list by the plans customers were on… so we could easily compare the responses of customers on the entry-level plan to those on the mid- and higher plans.
That’s why you don’t want to send one lump survey if you can help it. Because you should be targeting the prospects that are best for your biz – not everyone who might throw a dime your way.
We sent 1 survey to 4 segments. We only asked questions that would help us write copy, and we recommend you don’t waste your time on a full-scale ‘market research’ survey if all you wanna do is use the results to write better copy. (Steps 1 and 2 will help you know what Qs to ask. So will reading other blogs from our library)
The questions we asked included:
a) What was going on in your world that caused you to seek out a solution like Crazy Egg?
b) Give 1 example of something unexpected Crazy Egg has shown you that’s helped you optimize your site
c) Which of the following best describes your current role or title?
5. Analyze the Results of the Survey
Don’t just look at the survey data. Don’t skim it. It only helps you if you break it down into manageable parts. We create a research summary report for every client we take on; for Crazy Egg, that report was divided into these parts (i.e., this is the table of contents):
All the research we did made its way into this report. This can be a pain in the ass to fill out, but TRUST ME, it will pay off endlessly if you do it. If you’ve ever wished you could just write faster, THIS IS HOW YOU DO IT! Create a summary report once, and use it for the next 18 months.
Naturally, your research summary’s table of contents won’t necessarily look like this, but the key parts to take away are that you ought to finish with:
i) A proposed messaging hierarchy… based on data
ii) An understanding of what to say about your features, and how to say it… based on data
iii) Words to use (especially adjectives) to explain your solution… based on data
iv) A brief profile of whom you’re targeting… based on data
Students of our Conversion Copywriting Course [retired] spent 4 weeks creating their own report with their own data. For updated courses, visit Copyhackers’s school.
6. Let a Few Users Fill in the Messaging Gaps
For Crazy Egg, I knew we weren’t changing the look and feel or the general design of the page… so I wasn’t looking for UX or UI feedback from site visitors. But I did want to know how visitors were responding to some of the messages they were seeing on the page…
So we booked 5 sessions on UserTesting.com. (Don’t order more than that initially; you probably won’t watch more than 5.) Admittedly, paying for non-organic users to visit your site and talk through what they’re seeing is not the ultimate solution; it’s good, but, like all paid user research, it doesn’t net natural responses. But that’s okay. We’re just trying to learn right now, and we’re going to test our assumptions.
In watching people use the home page in particular, we noticed:
– A lack of interest around the same points that Crazy Egg’s scroll mapping had shown waning engagement, which started right about here:
– A belief that Crazy Egg offered actual eye-tracking, not an alternative to eye-tracking
– Users commenting on the value of the video but not watching more than 8-10 seconds of it; lack of viewing could be to save time in the paid session recording
– Users zipped by the text-heavy, image-light zones
7. Interview Current and Past Customers
If you’ve never sat on the phone with someone that uses your solution, your competitor’s solution or your client’s solution, you probably think this sort of exercise is worth skipping…
…Because it’s not until you talk to a user or past user for an hour that you see how unimaginably useful it is.
Some interviews are better than others. But the more you talk to people who are a) brand new users, b) long-time users and c) past users / refund requesters, the more you’ll be able to glean uber-rich insights into:
– EXACTLY the pain they expected your solution to neutralize for them
– The happy surprises they’ve found while using the solution
– How they describe the solution (so you can swipe their words)
– The words that keep occurring naturally (which is where the copy gold is)
Those are the 7 steps we took to move from staring at a page that was a proven winner… to beginning the process of creating treatments worth testing.
The Treatments We Proposed to Crazy Egg,
Which They Then Tested
Lance keeps meaning to do a post on “exclusion testing”. Hopefully one day he will. (Lance, that’s a hint.) With the Crazy Egg treatments, we were really interested in excluding elements and then figuring out how to improve on the ones that remained.
While I was doing research, Hiten conducted an exclusion test in which he took away 1 element: the Johnson Box. Which means he deleted the yellow-colored box to the left here:
Why? Because I wanted to see how much that box, which was taking up million-dollar real estate, was impacting conversion. The exclusion test showed flat results: the page converted the same with or without the Johnson Box. So we pulled it.
Thanks to the heatmaps, we knew what design elements were getting the most clicks. So, from there, it was all about the messaging. I created 4 variations of the page, which you’ll have to squint to see below. The first 2 hold truest to the original layout, and the last 2 veer further away from the Control:
The winning treatment, with a statistically confident lift of 13%, was… Variation D, the treatment on the far right in the image above.
Here’s its hero shot:
Where do winning messages come from? Your customers and visitors:
- The winning headline was pulled verbatim from a simple, catchy phrase a handful of survey respondents had used
- The 3 subheads were created based on the top 3 answers to our survey question: What was going on in your world that compelled you to look for a solution like Crazy Egg?
- The new opening shot for the explainer video was proposed to lead with a message, not an unfamiliar graphic; check out how video pauses/stills brought a 109% lift
- Instead of a “watch this!” caption for the video, we used a line of copy based on what users say they use Crazy Egg for: “to validate our design decisions”
- Countless survey respondents and interviewees said they used Crazy Egg to “optimize our user experience”… so on the page it went!
- On finding that the prospects most likely to pay for a mid-tier or higher plan were not startups but digital marketers, web managers and designers, we listed the 4 groups Crazy Egg is best for
…And that’s just for the hero section! The majority of the copy below the fold – which you can see by visiting Crazy Egg today – was also pulled from the messaging summary report I mentioned earlier. Here’s a snapshot of my favorite part of the page, which lists out 30 amazing things you can do with Crazy Egg in your 30-day trial (most of which came directly from customers):
Why did this page work (better than the Control, at least)? Because winning messages don’t come from you… and they don’t come from writers… and they don’t come from your boss… and they don’t come from old-school direct response rules. Winning messages come from customers and prospects.
Stop guessing. Stop writing.
Start listening. Start converting.
As Neil pointed out, Crazy Egg has spent $250,000+ on optimizing this page. But your startup probably can’t afford that. (Yet!) So learn to do it yourself, without wasting time. Here are 2 easy options:
1. Subscribe for free by clicking right here (or entering your address below) to get new articles, new case studies, and invitations to non-sucky webinars
2. Click here to watch the case study for the Crazy Egg home page optimization… for free-zies
CHALLENGE FOR YOU!
Every winning treatment can be improved on. So it’s your turn to improve on it.
If you were assessing this page before optimizing it, what would you consider testing?
Share one (1) fab idea in the comments below…