How I Optimized the Crazy Egg Home Page

Last week, Neil Patel wrote about the optimization of the Crazy Egg home page, which his biz partner Hiten Shah hired me to rewrite…

I’d met Hiten when we were speaking at Microconf, and I was stoked when he challenged me to beat a Control that was hugely intimidating. If you read Neil’s post, you’ve got some ideas on the test and the outcomes – and now you can see how I developed the winning copy. So you can do the same on your site…

The Page We Were Trying to Beat

We were up against a strong incumbent. Like, super-strong. Like, you’ve heard about it because it was the very page that had won in this popular and much-discussed A/B test by the uber-smart folks at Conversion Rate Experts

Recognize the following graphic? If you’ve been in CRO or online marketing for more than 2 years, you probably do. It’s by CRE. It shows the Control they were against… and the [ultimately winning] Challenger they created:

Crazy Egg Home Page

The CRE Challenger brought in a 30% lift over the then-Control… and proved a major win for everyone who’s ever championed the power of long copy to convert…

Their Challenger became the new Control.

And it later became the one for me to ‘beat’.

Of course, the saying goes “Always Be Testing” not only because Alec Baldwin made “Always Be Closing” the catchphrase of every salesperson from 1992 onward… and not only because Bryan Eisenberg wrote the book on it… and not only because it’s always better to test than to guess… but ALSO because optimization is an ongoing process, so you should never stop testing

That the Challenger here beat the Control here does not mean that long copy is better than short copy.

When B beats A, that does not mean B will also beat C, D, E, F, G or H. That’s why we have to keep testing and avoiding wholesale conclusions, like “Long copy beats short copy.” Because, as you’ll see, that’s not always true…

So with that in mind, we wanted to test the messaging on the Crazy Egg home page. For the purposes of this test, it was important to keep the general layout the same – rather than using, say, a dramatically different design, as was used in the CRE test.

We were trying to answer:

  • Which messages are critical to conversion… and which might be ‘extra’ or non-impacting?
  • Is leading with a benefit or outcome better than leading with a feature (i.e., eye-tracking)?
  • Is the long-form hybrid style working here?–is the power of the long-form narrative coming through?
  • Are the long-form-y elements, like the Johnson Box, adding value?

Here’s how we answered those questions and others, and helped the Crazy Egg team see that long copy isn’t the only way…

The Exact 7 Steps We Followed
(Don’t Optimize a Page Without ‘Em!)

In conversion copywriting, Research & Discovery is everything. You can’t write conversion copy if you don’t know what your stellar messages are. So we spent most of our time on discovery (and so should you!).

This is the exact process we followed to find the messages and messaging hierarchy worth A/B testing:

1. Start with a Recorded Assessment of the Page

Turn on Camtasia or whatever screen-cap software you use – there are free ones out there – and record yourself assessing the page you want to optimize. Think aloud. Take as long as you need to. Ask questions aloud; tell yourself to make note of X, Y and Z; tell yourself to set up a Qualaroo survey that asks X question. Say everything. Record it. Mp4 it.

And then send it to be transcribed; we use and recommend Rev.

2. Print Your Transcript and Grab a Highlighter

We’re paperless here at Copy Hackers… except when it comes to this step. So we have to send our transcripts to Staples to be printed on good ol’ fashioned dead trees. But it’s worth it. (Sorry trees!) Once you’ve got your transcript in hand, you sit down with an old-school pack of highlighters and:

- Use yellow for great copy ideas / insights

- Use pink for great design ideas / insights

- Use green for further research ideas

How do you know if something’s great enough to be highlighted? You’ll probably have waited 1-2 days for your transcript… so your original assessment isn’t fresh in your mind. That’s a good thing. Only highlight the copy and design ideas that jump off the page at you.

Highlight all further research ideas. They are all good.

3. Use Crazy Egg

Yes, we used Crazy Egg to optimize CrazyEgg.com. It was very postmodern. I dug it. Although all 4 maps Crazy Egg creates were useful, I most wanted to see the scroll-mapping… because a long page is tough to get through, and we needed to see what points were capturing and recapturing attention. The whole scrollmap is huge, so here are snippets of the more meaningful parts…

SMOKIN’ HOT AT TOP OF PAGE…
THEN THE DEEPFREEZE BEGINS (AND LASTS)

Hot top of page

AFTER EXTENDED COLD SNAP,
GETS WARM-ISH AGAIN AT BOTTOM (PRICING)

Cold until bottom of Crazy Egg page

4. Do a Segmented Customer Survey

The more you can segment the survey, the better. In the case of Crazy Egg, Hiten was able to segment his list by the plans customers were on… so we could easily compare the responses of customers on the entry-level plan to those on the mid- and higher plans.

That’s why you don’t want to send one lump survey if you can help it. Because you should be targeting the prospects that are best for your biz – not everyone who might throw a dime your way.

We sent 1 survey to 4 segments. We only asked questions that would help us write copy, and we recommend you don’t waste your time on a full-scale ‘market research’ survey if all you wanna do is use the results to write better copy. (Steps 1 and 2 will help you know what Qs to ask. So will reading our blog every week)

The questions we asked included:

a) What was going on in your world that caused you to seek out a solution like Crazy Egg?

b) Give 1 example of something unexpected Crazy Egg has shown you that’s helped you optimize your site

c) Which of the following best describes your current role or title?

5. Analyze the Results of the Survey

Don’t just look at the survey data. Don’t skim it. It only helps you if you break it down into manageable parts. We create a research summary report for every client we take on; for Crazy Egg, that report was divided into these parts (i.e., this is the table of contents):

Crazy Egg research summary for copywriting

All the research we did made its way into this report. This can be a pain in the ass to fill out, but TRUST ME, it will pay off endlessly if you do it. If you’ve ever wished you could just write faster, THIS IS HOW YOU DO IT! Create a summary report once, and use it for the next 18 months.

Naturally, your research summary’s table of contents won’t necessarily look like this, but the key parts to take away are that you ought to finish with:

i) A proposed messaging hierarchy… based on data

ii) An understanding of what to say about your features, and how to say it… based on data

iii) Words to use (especially adjectives) to explain your solution… based on data

iv) A brief profile of whom you’re targeting… based on data

Students of our Conversion Copywriting Course spend 4 weeks creating their own report with their own data. Registration for the course opens again in Sept 2014

6. Let a Few Users Fill in the Messaging Gaps

For Crazy Egg, I knew we weren’t changing the look and feel or the general design of the page… so I wasn’t looking for UX or UI feedback from site visitors. But I did want to know how visitors were responding to some of the messages they were seeing on the page…

So we booked 5 sessions on UserTesting.com. (Don’t order more than that initially; you probably won’t watch more than 5.) Admittedly, paying for non-organic users to visit your site and talk through what they’re seeing is not the ultimate solution; it’s good, but, like all paid user research, it doesn’t net natural responses. But that’s okay. We’re just trying to learn right now, and we’re going to test our assumptions.

In watching people use the home page in particular, we noticed:

- A lack of interest around the same points that Crazy Egg’s scroll mapping had shown waning engagement, which started right about here:

Crazy Egg boredom on page

- A belief that Crazy Egg offered actual eye-tracking, not an alternative to eye-tracking

- Users commenting on the value of the video but not watching more than 8-10 seconds of it; lack of viewing could be to save time in the paid session recording

- Users zipped by the text-heavy, image-light zones

Use UserHue to learn what to optimizeBTW, to get feedback on your site content from YOUR actual users when they’re on your site, check out UserHue.com by Adam Jordens and Copy Hackers’s own Lance Jones

7. Interview Current and Past Customers

If you’ve never sat on the phone with someone that uses your solution, your competitor’s solution or your client’s solution, you probably think this sort of exercise is worth skipping…

…Because it’s not until you talk to a user or past user for an hour that you see how unimaginably useful it is.

Some interviews are better than others. But the more you talk to people who are a) brand new users, b) long-time users and c) past users / refund requesters, the more you’ll be able to glean uber-rich insights into:

- EXACTLY the pain they expected your solution to neutralize for them

- The happy surprises they’ve found while using the solution

- How they describe the solution (so you can swipe their words)

- The words that keep occurring naturally (which is where the copy gold is)

Those are the 7 steps we took to move from staring at a page that was a proven winner… to beginning the process of creating treatments worth testing.

The Treatments We Proposed to Crazy Egg,
Which They Then Tested

Lance keeps meaning to do a post on “exclusion testing”. Hopefully one day he will. (Lance, that’s a hint.) With the Crazy Egg treatments, we were really interested in excluding elements and then figuring out how to improve on the ones that remained.

While I was doing research, Hiten conducted an exclusion test in which he took away 1 element: the Johnson Box. Which means he deleted the yellow-colored box to the left here:

Crazy Egg Home Page with Johnson Box

Why? Because I wanted to see how much that box, which was taking up million-dollar real estate, was impacting conversion. The exclusion test showed flat results: the page converted the same with or without the Johnson Box. So we pulled it.

Thanks to the heatmaps, we knew what design elements were getting the most clicks. So, from there, it was all about the messaging. I created 4 variations of the page, which you’ll have to squint to see below. The first 2 hold truest to the original layout, and the last 2 veer further away from the Control:

Crazy Egg 4 treatments

The winning treatment, with a statistically confident lift of 13%, was… Variation D, the treatment on the far right in the image above.

Here’s its hero shot:

Crazy Egg copy test home page

Where do winning messages come from? Your customers and visitors:

  • The winning headline was pulled verbatim from a simple, catchy phrase a handful of survey respondents had used
  • The 3 subheads were created based on the top 3 answers to our survey question: What was going on in your world that compelled you to look for a solution like Crazy Egg?
  • The new opening shot for the explainer video was proposed to lead with a message, not an unfamiliar graphic; check out how video pauses/stills brought a 109% lift
  • Instead of a “watch this!” caption for the video, we used a line of copy based on what users say they use Crazy Egg for: “to validate our design decisions”
  • Countless survey respondents and interviewees said they used Crazy Egg to “optimize our user experience”… so on the page it went!
  • On finding that the prospects most likely to pay for a mid-tier or higher plan were not startups but digital marketers, web managers and designers, we listed the 4 groups Crazy Egg is best for

…And that’s just for the hero section! The majority of the copy below the fold – which you can see by visiting Crazy Egg today – was also pulled from the messaging summary report I mentioned earlier. Here’s a snapshot of my favorite part of the page, which lists out 30 amazing things you can do with Crazy Egg in your 30-day trial (most of which came directly from customers):

Crazy Egg home page

Why did this page work (better than the Control, at least)? Because winning messages don’t come from you… and they don’t come from writers… and they don’t come from your boss… and they don’t come from old-school direct response rules. Winning messages come from customers and prospects.

Stop guessing. Stop writing.

Start listening. Start converting. 

As Neil pointed out, Crazy Egg has spent $250,000+ on optimizing this page. But your startup probably can’t afford that. (Yet!) So learn to do it yourself, without wasting time. Here are 3 easy options:

1. Come see me speak at Copyblogger’s Authority Intensive

2. Subscribe for free by clicking right here (or entering your address below) to get new articles, new case studies, and invitations to non-sucky webinars

3. Click here to watch the case study for the Crazy Egg home page optimization… for free-zies

CHALLENGE FOR YOU!

Every winning treatment can be improved on. So it’s your turn to improve on it.

If you were assessing this page before optimizing it, what would you consider testing?

Share one (1) fab idea in the comments below…

Crazy Egg home page new control

~joanna

Do your biz a favor: buy these ebooks, master copy fast, sell more

  • http://www.iConversing.com iConversing

    Great post, much to learn from. Joanna, are hover maps useful? I feel attention map show time spent, but hover maps sometimes show one whole line in the heat map where the user just scroll the mouse?

    • Joanna Wiebe

      The more data you have, the better (usually!). Do with it what you will, knowing that it’s the extremely rare case that a single source of data will tell you the whole story. Triangulation is where it’s at!

      What a scroll, click, ‘attention’ or heat map can do is help you generate hypotheses on how users are interacting – or are not interacting – with a page. You can sometimes run with that data (e.g., use it to inform an A/B test), and other times you might want to see if user testers (e.g., with UserTesting.com) ‘validate’ your heatmap data before moving forward. Again, triangulate! :)

      I find the maps in Crazy Egg — especially Confetti View — extremely eye-opening.

  • http://www.designhandyman.com/ DesignHandyman

    Fantastic post, Joanna. I meant to reply sooner, but it was impossible to me. I really liked the insights you give in this post. Regarding the CrazyEgg page, I honestly feel their room from improvement in some parts.

    – CrazyEgg logo on top:
    I think it’s a bit too small. I would place it like in this page – https://www.crazyegg.com/overview – Notice the “Visualize Your Visitors” tagline. See what I mean at: http://screencast.com/t/xE4oATts3

    – What is the correct form to write the brand is it “Crazy Egg” (with space) or “CrazyEgg” (without space)? Marketing-wise, it would be good just as “CrazyEgg” (without space)

    – The CrazyEgg visualizations (Heatmaps, ScrollMaps, etc) have a logo and should be on the homepage (see the logos at: https://www.crazyegg.com/overview) – BTW, are these registered TM? Because I think they should.

    – The list of “…30 Powerful ways…” should be personalized to separate each line better and to improve readability.

    – I would place a single testimonial and the rest on rotation/navigation/tab to save space.

    – I found a bit odd to not have a proper footer, but maybe is just me.

    – On a sidenote, I think that CrazyEgg could built a database of case studies, showcasing different customers using CrazyEgg tools on different scenarios, and showing the “before-and-after” results their customers achieved. Some video interviews could work too to give a “human-feel” to the tools.

    That’s it. :-)
    -

  • Joanna Wiebe

    hahaha! I’m sure you did, Daniel — ‘cos you’re hard core! And, yes, there is still testing going on there. I think it’s safe to say that Hiten and Neil are very, very keen on testing.

  • Edrian Madraiga

    Hi Ms. Joanna,

    You have no idea how much I respect you now..to learn that you were hired by Hiten Shah, and been able to work directly with the Crazy Egg team!? Wow! I’d be over the moon if that was me!
    >CONGRATULATIONS for beating the control! You wicked smart! :) (I don’t know if any of the other commenters knew how MASSIVE this assignment is…but you actually did it!)

    I confess..I have signed-up for your newsletter, but I’m sorry that I haven’t really paid serious attention to them..never again. You’re now definitely one of my copywriting and CRO superheroes..!
    > I may be running my own profitable business already, but it would be an honor for me to get to work along side you — even as an intern for copyhackers! (would that be possible? ;)

    Also, thank you for your effort to make a case study this gooood. You’re awesome! :)
    -Edrian Madriaga

    • Joanna Wiebe

      …I’m going to go by “Ms. Joanna” from now on. ;) Thanks for your comment, Edrian.

  • http://www.blackbeltrobots.com/ Dennis Moons

    Thanks for the great post Joanna!

    Like Ray, I find it fascinating that a brand focused message, the name Crazy Egg is in the headline and sub headline, works that well. Perhaps something to do with the brand getting stronger?

    • Joanna Wiebe

      Yeah, or could just be the need to test something else. This headline talks less about the features than the control (i.e., “eye tracking”) does, and it’s speaking to what’s uniquely desirable about Crazy Egg… but that sure as Helsinki doesn’t mean there’s not tons of room for further optimization.

  • http://raymondduke.com/ Ray

    The headline “A Crazy Egg Picture Tells a Stunning Story” doesn’t make sense to me. It’s a type of headline that suggests someone knows what Crazy Egg is and does. I would test the headline.

    • Joanna Wiebe

      That’s totally true — we know it’s something that existing customers get because we stole the language directly from them. :) The assumption is that visitors who are similar to existing paying customers will ‘understand’ similar messages — but that is just that: an assumption. Further testing of the headline could and certainly will pay off. I think if you could find another swipe-worthy statement among the research — or do more research in search of such a statement — you’d have a strong challenger on your hands. A visitor survey could help glean test-worthy swipable headlines from non-customers.

  • http://webcopyservices.com/ Zafifi

    I would consider testing the text at the bottom of the page “Start your free Crazy Egg Trial Today…. adding something like ‘Start optimize your website from $9 per month’ – to match the message on the signup bar.

    • Joanna Wiebe

      Great call. The signup bar is new; it wasn’t part of the treatments I proposed, but based on all the tests I’ve seen running with it, it seems to be working…

  • Scott Miller

    The other set of variations I might try:

    If the heatmaps show a similar cold zone in the middle, is split testing reordering the content blocks. Faces and testimonials below logos and before features for instance.

    I might also reorient the features text to be value proposition based. Right now they lead with “Heat Maps” versus “Find What Works (and Doesn’t) on your Pages” and the subtext being “with our unique Heat Maps feature…” — or “Optimize for Specific Audiences” and the subtext being” with our unique Confetti feature, segment users by traffic source …” — which you could also use to target variations of homepages or landing pages based on where folks come from for overall conversion optimization… one size does not fit all!

    Finally at the very bottom there isn’t a call to action… I see it’s sticky at the top, but there’s nothing there to click — maybe have a repeat of the simple signup form from up top (or maybe an arrow that appears directing you up top to sign up or what to do to get started). You might also include a link from the “What Plan You Choose” to a pricing page unless you already know that pricing upfront reduces free trials.

    These are all niggles around the edges of what’s clearly an incredibly well done page that I’ve already taken notes on as we think about our own site… but as you said the point of diminishing returns is unknowable and often way further out than we think!

  • David Larsen

    Is the video frame meant to be a laptop or a massive TV? I might test a shiny device.

    • Joanna Wiebe

      That’s a great question! I didn’t want it to look like a laptop because that feels very SaaS-y / startupy, and we knew we weren’t directly targeting that segment of small- to mid-sized businesses. So I recommended the “TV” — it doesn’t scream “software”, but I think it does say “this is a thing you watch stuff on”. :) A test of it could be fantastic!

  • http://copygrad.com/ Will Hoekenga

    I would do some testing with the signup bar.

    Instead of relying as that bar at the top to be the sole place where people can sign up, I would throw in a few more “Show Me My Heatmap” buttons throughout the page that pull up a signup window (ala LeadPages new LeadBoxes feature).

    You could also then test totally doing away with the signup bar at the top and seeing how the buttons/popup signup work on their own.

    Great post, Joanna!

    • Joanna Wiebe

      Very smart, Will!

      • Neil Wesley Flinchbaugh

        Along those lines, I wonder if they’ve tested different variations of the signup bar.

        I think it’s great that they ask for just an email and password to sign up (minimizing friction). However, for some reason when I see “Email” and “Password” boxes together I assume it’s a login area for people who already have an account.

        (I’m probably used to longer signup forms that require me to verify my password, etc.)

        And the CTA copy “Show Me My Heatmap” doesn’t make it immediately clear that this is a signup, not a login button. There is a separate “login” link, but it disappears when you scroll down.

        Or is it just me?

    • Neil Wesley Flinchbaugh

      Along those lines, have you tested different variations of the signup bar?

      I think it’s great that you ask for just an email and password to sign up (minimizing friction). However, for some reason when I see “Email” and “Password” boxes together I assume it’s a login area for people who already have an account.

      (I’m probably used to longer signup forms that require me to verify my password, etc.)

      And the CTA copy “Show Me My Heatmap” doesn’t make it immediately clear that this is a signup, not a login button. There is a separate “login” link, but it disappears when you scroll down.

      Of course this is nitpicking in what is a tremendous page. Great post Joanna!

  • Cameron Bayly

    Your articles are always an easy read and full of useful ideas. Thanks again Joanna!

    • Joanna Wiebe

      Thanks, Cameron! :)

  • http://www.radianttribes.com Josh Bowers

    Does crazy egg do split testing as well as user behavior tracking? Or was something else used to split up the traffic to the different variations?

    • Joanna Wiebe

      Hiten used a split-testing tool to run the tests. Crazy Egg isn’t a split-testing tool, no; it helps you find ideas worth testing, validate untested design changes, etc.

  • Scott Miller

    If there’s a positive correlation between folks watching the video and converting, I’d consider changing the video thumbnail to encourage more video views. Specifically the image on the right is of a male with a moustache that makes him look like he’s frowning, and he looks passive and not like a participant. I would try an image of a person who is (1) looking at the chart itself [drawing the visitor’s eye to it and wondering what’s so interesting] and (2) clearly delighted (smiling, arms outstretched, or in a victory V) and (3) having an epiphany (tired lightbulb over head metaphor or some such) and (4) female [my experience suggests they drive more conversion but your mileage may vary of course]

    Note that the first part is of course critical… do video watchers convert at a higher rate? I have had personal experience with including a video thumbnail of a good explainer video at key conversion moments and seen uplifts in conversion *irrespective of whether or not visitors actually watched the video*!

    • Joanna Wiebe

      Great point about the opening shot / still / thumbnail. Testing variations on this can be very revealing (see my note and link above about Lance’s cool video A/B test).

      As for whether video watchers convert at a higher rate, in the case of Crazy Egg, Conversion Rate Experts found that a new explainer video helped conversions by something like 65%. (See the link to CRE’s case study above.) We didn’t change the video for this test.

      • Scott Miller

        Certainly! my side point at the end was that it might also be interesting to look at from the other end of the funnel — what % of those who converted actually viewed the video? In our case just having a thumbnail of the video increased conversion, BUT by a far larger number than the count of people who actually watched the video! Just its presence must have lended an air of credibility and comfort… It honestly makes me wonder whether putting a placeholder for an explainer video that links to “coming soon” might provide people on a shoestring budget an early conversion lift… even though it’s a bit disingenuous for those who do try to watch it :D

  • Beatrix Willius

    Great article. This is really fascinating.

    Ideas for testing:
    – which story do the heatmaps tell? Perhaps something more about website redesign or conversion optimization in the subheadline instead of “need a snapshot of user behavior”, which I find a bit bland.
    – The brown on yellow doesn’t pop enough. The font of the button doesn’t look good on IE.

    • Joanna Wiebe

      Interestingly, Beatrix, that brown-on-yellow button has been impossible to beat thus far. The only difference between Variation C (not winner) and Variation D (winner) is the button; Variation D keeps the brown-on-yellow button of the CRE Control.

      But it could be very interesting to keep testing that, yes!