Presented live on Tuesday, June 26, 2018
Register for our live tutorials
How’s your copy working … or is it? Watch this tutorial as Ashley Greene of UserTesting.com shows you how user testing can help find – or refine – your messaging.
**Bonus Content**
Get the free template for Analyzing Usertesting.com messaging tests.
Connect with Ashley, Twitter:@ashleykgreene
TRANSCRIPT:
Ashley Greene: Nice to meet everyone.
Joanna Wiebe: I know. I can’t wait just to have the office to do that in. Yes, cool, that will be fun to do, but today, today is all about Ashley showing us some cool ways that you can test how your copy is or is not working, identify problem messages, identify new opportunities, identify good messages that you shouldn’t delete, things like that. Opportunities for optimization, etc, but before we dive in there, and there is a lot planned, and I know people are still filing in, before we dive in, we have some quick housekeeping.
Joanna Wiebe: Recording, we are, yes, we’re recording this tutorial so you’ll get the replay in case there are things that you’re like, wait, I want to see how Ashley did that again. There will be a few links to templates and other things that you might want throughout this training, so stay on and we’ll Chat that over to you today and Chat is for saying hello, and thank you people who are already Chatting over and saying that my kitchen is gorgeous, thank you.
Joanna Wiebe: That’s because I worked so hard to get here, but I’m so glad. Okay, and did I promise cookie? That’s funny, Joey’s messing with me. Yeah, we’ve got chat going on, people are already chatting. If you have questions for Ashley that you want her to answer, please put those in the Q&A area, and then we’ll get to those at the end of the tutorial. We’re already five minutes in and I know Ashley has a lot planned. I’m going to mute myself, which is always hard for me, and Ashley, let’s hear it. What are you teaching us today?
Ashley Greene: Okay. I’m going to teach everyone, we like data. We love copy, but often times it takes awhile to get the data, so we either wait too long, or we don’t get it at all. I’m going to show you how you can use user testing to really refine your messaging. Not so much, although it’s actually great for it to find messaging, but we know, you know, research and all those things Joelle teaches all the time are really what’s great for that. But what I say is good messaging is great, bad messaging is catastrophic. User testing is absolutely fantastic for that.
Ashley Greene: I’m just going to kind of quickly share my screen with you guys and go through some things. I’m going to go really fast, but you’ll have a recording, so you’ll be able to go back later, and any questions, please, of course, just put them in the slide show. Basically, you can kill, conversion killing messages in 24 hours user testing. Test copy before you send it to clients, who wants to send conversion killing copy to clients? No one. And if you’re at a start up, reduce your endless cycles of internal iteration, and just chatting, or if you’re a freelancer, you know, you’re doing it on your own, get some real data from your real users, the customers, fast.
Ashley Greene: Keeps in line with move fast and break things, everyone’s familiar with that. Also, diagnosing a bad experience right away. Today it’s going to be super hands on, super actionable, that’s what a Tutorial Tuesday is. We’re going to go through an exact testing protocol, specifically for testing home page messaging. How to select your audience, you know the questions that I use, all of those things. I’m not going to necessarily teach you about unbiased questions and all that. I’ll fit that in, but I’m just going to actually just give you a great question set that you can use.
Ashley Greene: I will warn you, user testing is absolutely addicting.
Joanna Wiebe: It is. So true.
Ashley Greene: It’s like, once you start and you realize how quick and easy and cheap it is, you will get addicted.
Joanna Wiebe: I would just add, can I add, Ashley?
Ashley Greene: Please add.
Joanna Wiebe: I have a hard time muting myself, but …
Ashley Greene: Please, please, don’t mute yourself.
Joanna Wiebe: Don’t send your user testing.com access or videos to your clients, because it’s addictive for them, too, and in my experience, they start to have new questions that you don’t even want them to be exploring. Keep the user tests to yourself, in my experience, I would say.
Ashley Greene: I agree with you. I like to include raw data when I do any type of work like that, but the downside is, if people go through it, sometimes they can hone in one specific comment, or one specific thing, and kind of just go down this huge rabbit hole.
Joanna Wiebe: Totally.
Ashley Greene: I agree, especially if you’re working with clients, you know, your job is to parse that data for them and give them the actual insight. Really sell that job to be done, I 100% agree, because otherwise there could be a rabbit hole you really don’t want to go down.
Joanna Wiebe: Yes.
Ashley Greene: Just a couple of myths, because talking about using user testing, we talk about it for product, people think about it for UX, UI, prototyping, all of those things, and absolutely, user testing, there’s a couple different, there’s Usability Hub, there’s a few different ones, but today we’re going to use UserTesting.com, it’s accessible to everyone, even if perhaps they need to user test their own website, and I’ll show you why. It’s not just for product. It’s great for messaging and copy, and specifically, killing that conversion killing copy fast.
Ashley Greene: You don’t need to be a real researcher. I’m giving the formulas, don’t worry about that, and it doesn’t take that long. Turnaround time, I did two for [Air Story 00:05:06] last night, I got them in less than an hour, and I added a specific qualifier for copywriters, okay? Fast.
Ashley Greene: Use cases, messaging copy as we’re talking about, it’s great for user onboarding, for prototypes, even email feedback. We just put the email in a link, it’s fantastic. Checkout, Ecom, sign up flows, UX, UI, specific high impact pages like pricing. There’s a million one use cases, I swear you will really get addicted.
Ashley Greene: Just to kind of quickly emphasize this point, this is kind of like an old, kind of fable, I’m not sure if you’ve heard about it before, but Salesforce, this is way back machine from 2011. This is what their website looked like. Their H1 was No hardware. No software. No headaches. Seems pretty good. They ran, they had an idea that they wanted to be more about a company that did good, so they went forward with a big H1 change. They did roll it out US only, so they didn’t go worldwide, but US only, and they changed it to this.
Ashley Greene: Now, welcome to the social enterprise. They weren’t able to tell right away anyone who knows, we know analytics, there’s a lot of different factors for things, and if you’re basing it on not just on visitors or necessarily top of funnel sign ups, but customers, it can take awhile to become apparent something’s wrong, and that was kind of the case here.
Ashley Greene: What happened? They saw something was going wrong, they saw at least a 10% drop in leads. Now, for Salesforce, even in 2011, that’s huge. What did they end up doing? They ran a five second test, which I’m going to show you how to run, and what happened?
Ashley Greene: Salesforce, and the biggest thing about messaging, once you’ve done your research and you’ve done the writing, is we all know it’s subjective, so really disfusing a confusing experience or bad messaging can be catastrophic, and that’s really the focus today. Salesforce thought they were a social enterprise. We care about more than making money. What do testers think? They thought they were software for nonprofits, and they bounced.
Ashley Greene: Five second test. Couple hundred dollars, this is, we’re talking about, I don’t know the exact number build, their old CMO did a great podcast on Humans Strike Back, and he got into more details there than I’d ever heard before, but I think it’s safe to say hundreds of thousands of dollars, if not more.
Joanna Wiebe: That’s crazy. I just want to pause again, yup, I’m interrupting, sorry. I just want to pause for a second on this, because in the Q&A, Carrie asked, I’m sure others will ask, but how do we get budget for user testing? I think it’s how do we get budget, but yeah.
Joanna Wiebe: This case study, this, right? This is a dramatic example, but it is, it cost so little to run these tests that could really save your butt. Everyone in the boardroom can agree that a message came from the right place. It’s on brand. It sounds right. Data driven, wait til you put it out there, and this is the thing to use to support getting budget, so I just wanted to throw that out there.
Ashley Greene: Absolutely, and that’s why I have like, you know, lots of small examples, and other big examples, but Salesforce is just such a huge example that I know everyone can recognize and get buy in. Think about they had the best marketing teams in the world, and they still do, and this is what happened.
Ashley Greene: We’re going to talk about this a little bit later, but people think they need a lot of user testing videos. You really, really, really don’t. For each iteration, three to five videos is all you need. We’re talking like $250, USD. I’m in Canada, but USD, that’s it. It really isn’t a lot of money, and you don’t need a lot of videos, and it doesn’t take that much time. That’s exactly why I included those stats. 10%, don’t know the exact amount, but hundreds of thousands, maybe millions of dollars lost in sales in that messaging test.
Ashley Greene: Key point we’re going to use for user testing, confusing messaging is catastrophic. We all love good messaging, but even more importantly is bad, confusing messaging.
Ashley Greene: So, let’s go. I’m going to kind of go back and forth just because of the nature of user testing, to show you some things. I think I need to user test, no offense, if anyone from user testing is on here, their own webpage, because it’s really hard to figure out how to sign up an account, if you go to press free trial, it just doesn’t make sense.
Ashley Greene: Here, what you can do, if you don’t have an account already, just go to pricing. You can, if you want to get an enterprise account, that’s different, but just right away, you can just start. Go order now, fill this in, you have an account. I’m just going to log in. I created a separate account for this. My background’s law so I’m a little bit paranoid about confidentiality of information.
Ashley Greene: I’m going to show you guys how to start a new test. I did a couple tests here, which I’ll show you a little bit after about how you can do some analysis, but we’re going to start a new test.
Ashley Greene: Out of curiosity, if you want to, you can always create, duplicate a test here, which makes things really fast, but we’re going to go to a new test. Now, this is a screen, when you first sign up, this is what you’ll be put to right away. You can test a prototype, basically any file you can upload or prototype URL. Envision files, Photoshop files, anything like that, apps, but we’re going to focus on websites because we’re focusing on messaging.
Ashley Greene: Choosing the target market. I always like to name my tests. CCCHTT. I always like to put the date that we ran the test and the purpose of the test because if you start running more tests, it’ll get confusing, so the more details in your tests the better, you get a thumbs up. As we talked about before, we don’t need a lot. It’s better to have smaller testing cycles, because even the best questions in the world sometimes always need to be tweaked and things like that, and you really don’t need a lot to see some big patterns. If you need more, I can always add more, but generally, three to five and you really start to level off it more. Really, it can keep it small. We’re going to go down to three here.
Ashley Greene: Now, depending of course what you’re user testing, I did it using Air Story for this example. We would think about what device you want to test them from, so we’d pick computer, but you can do tablet and smartphone as well. I recommend the mobile recording method, and then we can just go through, these are our selection factors. For demographics, again, we’re using Air Story as an example, but you would use your own.
Ashley Greene: I just picked 25 to 50 … too. Income, if you’re a copy writer, that’s the audience for Copy Hackers, probably making a little bit of money. I just went 40k to 150k. Gender, you know, I just kept it neutral but of course, you can pick. Country, I picked US because that’s where the majority of traffic came from. Web expertise, so this one’s interesting. If we’re talking about SaaS, software, you might be inclined to go with advanced web user, and if you’re talking about something developer specific, I would, but it’s probably better to go with average web user, because you really want to test for confusing, really plain issues.
Ashley Greene: You can do both, ideally you can do double, but I actually like to, unless it’s a highly technical product, use average web user. You can pick your operating system. I’m just going to keep this broad, web browser, I think I chose Firefox, or I think I chose Chrome because of the Chrome extension that I love for Air Story copywriters. A lot of them are on Facebook, from my understanding, so I chose use Facebook, but you could keep it really broad, and you could have … I don’t really worry about this too much, and if you wanted to, because you want to test a couple different segments.
Ashley Greene: Maybe it’s, you want to test computer signups, desktops versus smartphone, you can just quickly add another group right there, if you want, and it will kind of duplicate it, and that’s a really easy way to do that. Just going to delete that. We have it, and then we can have a screener question, and this basically, do we want to talk to just copy writers, or the average population?
Ashley Greene: Now, there’s pros and cons for both. Obviously our market’s going to have knowledge and things like that, however, a lot of messaging dies and kills conversions because it’s just confusing. It’s full of jargon, even if we don’t see it that way. It’s really nice to get a balanced perspective of both. If I had to pick and choose, I would definitely go for the more specific group, which is what I did for the test.
Ashley Greene: Just show you here, be really careful with how you write your screening question. I’m actually not even 100% in love with how I wrote this one, but try to be as specific as possible. Get a few people’s opinions. For this one I said, “In the last 12 months, have you been paid as a copywriter? Either as an employee, or through your self employment,” this is to capture anyone who’s made any money, basically, as a copywriter.
Ashley Greene: Answer yes, I’ve been paid to do copywriting in the last 12 months, I’m going to accept them. No, I’m not a copywriter, reject. Really be really picky about your screening questions.
Ashley Greene: All right, creating a test, we’re going to go back now. I’m not going to put the screening question in. I showed you what it would look like, just too long to write it all out. This is where we put in the website and give a check. Now, the introduction, the mindset users should have, again, if you’re going with a broader, more general population, not copywriter specific, I would say, “imagine you are a copywriter who does this and this and this,” but for more general tests like the one we’re running, “You’ll be answering questions [inaudible 00:14:49] about a homepage and website, please read all questions out loud. Remember to share your thoughts out loud as you answer the questions.”
Ashley Greene: That’s a really good way to get started. Now, to questions. I’m going to do a mix of showing you on here, and in the PowerPoint just for time. My favorite, and this is the one that literally caused Salesforce to figure out that they’re telling people, or the majority of people that they were selling software to non-profits is the famous five second test. It’s already preloaded here. You can’t change it, so you can just drag and drop it. It’s really easy, and there you go.
Ashley Greene: You have a couple different types of things. There’s a pro subscription, and a non-pro subscription. User testing, I have an old account, but they’re not as straightforward with their pricing as I would like. I tried to call last night for you guys to get some more information, but you can do it. I’m using verbal responses, because I have the pro, but you can also just use blank tasks, they also have View Popular Tasks, which are great and you can just drag and drop them. If you’re ever stuck, see examples from others. They’re really helpful with their onboarding and UI, which makes sense given that they’re UserTesting.com.
Ashley Greene: Blank tasks basically are really open ended. If you’re using a blank task, you want to make sure you remind the person to speak it out loud versus a verbal response which automatically, they’re prompted to speak it out loud. Because it’s on messaging, we’re not getting them to go through flows and things like that, we’re just honing in, so it’s really about verbal responses. Pretty easy, verbal.
Ashley Greene: Okay, we’re going to go through the twelve questions to ask. We have five second test, which came preloaded, and then you can, just to show you what a blank task would be, I’m just going to copy and paste this, it’s literally drag and drop. Don’t worry about this, that’s how easy it is. Now we’re going to go back to question three, basically without scrolling, what are your initial impressions?
Ashley Greene: The reason I do that is I really want to test the clarity of our H1 and our value prop, which is almost always above the fold, and if it isn’t … probably good idea to put it there. Now we’re just going to go straight into verbal questions, and I’ll show you how to quickly do that. Just drag and drop, that’s literally it. I’m going to go through the rest of the questions on the PowerPoint, just for time’s sake.
Ashley Greene: Question two we talked about. Without leaving the home page, what are your initial impressions? The first time, we didn’t let them scroll, and now we’re just focusing on the homepage, and then without leaving the home page, in your own words, what do you think … It’s really important if you’re asking them to recall messaging, to ask them to do it in their own words. You don’t want them trying to copy back your words to you. It’s always good to put that reminder in there. What do you think this website/name can do for you?
Ashley Greene: This website is a little bit better versus name, because … but, if you want to be a little bit more specific, you can there. What questions do you have? You don’t have to ask in your own words, because you’re just asking them about questions. What questions do you have after viewing this homepage? Again, it’s really important when you write these, that you don’t write bias in questions. That’s one of the biggest mistakes people make.
Ashley Greene: I’m not going to have time to get into that lovely thing, but think about if you watch any legal shows, if you ever get, you see objections for leading the witness, it’s very much like that. That’s why I’ve italicized “If any?” Because you don’t want to imply they should have questions.
Ashley Greene: Is there anything confusing about this homepage? Be specific. Reminding people to be specific is also a great tip. What do you think is missing from this homepage, if anything? Again, trying not to lead, be specific.
Ashley Greene: In your own words, what do you think this website sells? Please be specific. Now, if you wanted to be as non leading as possible, you could say “What does this website offer?” However, there’s a little bit of, I think the probative value outweighs the prejudice here, to ask what it sells, if we’re doing it for a piece of software.
Ashley Greene: In your own words, what do you think you can do by signing up for this website/product/service? Please be specific. Again, I wouldn’t call it by name yet. We’re going to get to that.
Ashley Greene: What three words would you use to describe this website? Please explain your answer. Then finally, now that we’ve gone through all of the other questions without leading the witness, because sometimes they may not even know what your product is called. Believe me, that happens. In your own words, what is X? That is your product, your service, your company, whatever it is, in this instance, AirStory.
Ashley Greene: Then lastly, this is great for positioning your messaging and just compare to really great information, do you know any companies/products/services/website, keep it really broad. We don’t want to lead, that are similar to this one? If so, explain why.
Ashley Greene: If you follow these 12 questions, you will get some really, really great insight on bad messaging. On good messaging, other insights you can share with other teams, but these 12 questions, you can’t go wrong with them for testing homepage messaging. The big thing, and I want to show you the written responses, which for some reason, are there.
Ashley Greene: At the end, this is where they go through and they do everything verbally. After they’ve done their test, they can ask kind of follow up questions. There’s four that I really like for this. The first is, basically now that you’ve had time to explore this home page, what do you remember best?
Ashley Greene: Basically, the first three questions are repeating the five second test at the end, to see how their response changed from beginning to the end. Those three questions are basically identical to the five second question, test questions we asked at the beginning, and I really recommend that, because it’s a good measure to know that, because do we know user behavior often? If they don’t know anything about you, they’re going to give you a quick skim, at best 10 seconds. You really need to know if you’re making that right impression first on. You’ll be surprised at the difference. And lastly, if you had a magic wand, how would you improve this website? That’s always just a fun one to throw in.
Ashley Greene: The conclusion formula. Analyze, basically, the big thing is, and I’m going to show you a little bit quickly how to do that and have a template made for you, is you’re looking for patterns. If anyone knows what this is, it’s the Fibonacci Sequence.
Joanna Wiebe: Oh, I thought you were going to let them chat it over. I didn’t think that you knew.
Ashley Greene: Oh, I was going to, and then I looked at the time and I’m like, no! But yes, does any … sorry. Yeah, it’s …
Joanna Wiebe: That’s funny.
Ashley Greene: Sorry, is someone going?
Joanna Wiebe: No, no, nope, nope. Go ahead.
Ashley Greene: Basically, you’re looking for trends in patterns, that’s it. If you’re familiar, I should have let you guess. I was just looking at the time, this is the Fibonacci Sequence, it’s a famous pattern in nature, in mathematics, and ever watch Da Vinci Code, it’s in there as well. That’s basically it. Look for patterns in your videos. You only have three to five. It’s not that many to get through, but of course, I know you want some more specifics, I’m going to spend another five minutes just quickly showing you some analysis, and what your user testing video looks like.
Joanna Wiebe: Excellent.
Ashley Greene: Awesome, all right. We go through, we create this test, literally you just go with the order, just so you know there’s a problem with my order because I didn’t finish doing it. Put credit card in, schedule test, literally, you will have something in less than 24 hours. Okay, I’m going to go to my dashboard. I’m going to look at my current tests, and remember, duplicating tests is really easy. You can just do it right here. I’m going to use this one. You can find out more information about your test right here. Yes, okay. Let’s go to this test.
Ashley Greene: I’m actually just going to play, this is exactly what you get. You have notes and clips. This is really good for things you want to investigate or come back to later. I don’t use it that much, unless there’s just a really big point. I’m going to do everything in the template, you can look at your tasks. If you have some premium subscriptions, you can get a full transcript. You really don’t need them the way I’m going to show them to you, although I am a fan of transcripts. If you don’t have that feature, all you have to do is open QuickTime, do an audio recording of the recording, send it to Rev.com. It’s a dollar a minute, and get transcripts that way, but you really don’t need it, and then your written answers are going to be here.
Ashley Greene: I just like to use notes and clips for when we start.
Recording: [inaudible 00:23:10]
Ashley Greene: That’s how quick the five second test is. It’s fast, but sometimes, you know, user behavior, that’s all you’re going to get to make an initial impression. It’s tough, but it’s tough because it’s hard. With this, you can succeed at it. I made you guys a little template based on our questions, to make your analysis really easy, there’s a little legend here. Basically, we’re focusing our analysis in killing bad messages versus coming up with fantastic messaging the first time around, but you will find information on all of them. Let’s show you quickly how I would do this for the five second test.
Recording: [inaudible 00:24:16]
Ashley Greene: All right, it’s next question. That’s literally, you can watch the video first. I recommend, just go through it and do your analysis live like i have literally question for question, I have this all lined up for you. You just kind of go back and forth. If you need to rewind, you have every five seconds. Right here is literally every single task, so it’s time stamped. You can kind of go back and forth.
Ashley Greene: I do have a bonus for you. I’m just going to show you how I go through and analyze this whole thing. I already recorded it this morning, but the sound wasn’t working, and you’ll see I have quite a bit of back and forth, and generally, if it’s a ten minute video, also, by the way, a good user test is between 10 and 20 minutes. You definitely don’t want to go over 20 minutes. If so, you need to really pare down your directions to make sure your time limits are clear, if you’re finding that’s happening.
Ashley Greene: This will take about 20 minutes in ours, you know. Just always kind of double it, and that’s really safe. The other thing, when you’re going through, I want to show you guys quickly, is you’re really looking for patterns. When you go through, the second time, the third time, if someone else mentioned this, or they said, “Oh, it’s AirStory.” Doesn’t have to be verbatim. When you’re writing things down the first time, I recommend trying to get as verbatim as possible, but when you’re looking for patterns, it doesn’t have to be exactly. They could have said, you know, the company’s called AirStory, their software’s AirStory. It’s AirStory, so I would have put down twice, if that was the second test.
Ashley Greene: Let’s go back and watch a little bit more.
Joanna Wiebe: Of course for us, we’re not hearing the audio, but if we were …
Ashley Greene: Oh, you’re not hearing the audio?
Joanna Wiebe: No, no.
Ashley Greene: Oh, I had no idea.
Joanna Wiebe: It always happens with like, tech.
Ashley Greene: Oh my goodness, I’m so sorry.
Joanna Wiebe: No, no.
Ashley Greene: We won’t do any more of that. I thought you could hear the audio. I’m so sorry.
Joanna Wiebe: That’s okay.
Ashley Greene: I will, as a bonus, do the whole analysis, and show you exactly how much I go back and forth and how I go back and organize things afterwards, but generally, this makes it really easy to spot patterns. You’ll notice, I did put the written follow up to the five second test up close to the top, even though that’s going to be at the end, because you want these two things close together for comparison afterwards, and then three is when we start the regular section of the user testing. Makes it really easy, and then there’s last, I know the order’s a little bit weird, but in the end it actually works out better.
Ashley Greene: This is that last written follow up that had nothing to do with the five second test. Other observations, insights you have, just kind of bullet them down. Then I like to, after I’ve done a test, write two to three sentences, just of my thoughts. That’s it. Just very quick and dirty, and again, this whole process, you’re just going to … I like to have it, even if you like Google Sheets, it’s easier to tab, for me, having a Word Doc, and I just still prefer Word Docs, or AirStory if I’m writing content.
Ashley Greene: Plug for AirStory because it makes writing research content so much better. I have to get all my lawyer friends using it. Just tabbing back and forth between these two makes it so incredibly easy. Yeah. Without you guys hearing it, I think that’s probably a good place to end it, because we can’t hear it. It’s probably not that helpful. You can see it in the bonus content if you want, because I knew I wouldn’t have time to show you the whole thing, and the key is just, rinse, wash and repeat. Never do more than three to five videos, even if you want to do more, because doing more, you’re going to be more sure of the patterns, but again, you’re generally going to see them in three to five anyway.
Ashley Greene: Also, you’re going to see your tests. You can have the best questions in your world with this particular group. You need to modify them and things like that, tweak them, maybe everyone’s taking way too long on the test, so you need to cut the time limits on them, just in brackets. Always just do cycles of three to five. Three to five tests at a time, and you’ll be A-Okay, and you’ll be absolutely shocked at what you will learn from three to five videos. There’s not a website in the world that we couldn’t find something useful from.
Joanna Wiebe: Yeah, fantastic, that’s amazing. I was surprised going through that, you didn’t use the notes inside, like the note taker inside UserTesting.com, because that, I love that, but you don’t use it?
Ashley Greene: I don’t, because I’m really there, I’m looking for patterns, and it’s much easier to do it like this. Now, there was a point in this user test, for example, she was confused about something, and I wasn’t sure if it was going to prejudice some of the data, if I wanted to make some notes. I would have put a note there, to kind of come back and look over it if I wanted to. I use it for something that’s really confusing or big, or I know I just … I go back and forth and I use that five second back all the time, trust me, because you’re going back and forth, doing typing, trying to verbatim, and going back and forth, use that a lot.
Ashley Greene: I use it for that, but I think this way, again, I’m trying to be quick and dirty and get people doing this, because once you start, you will be addicted and your business and your company will be a lot better for it. Yeah, even myself, I like doing it this way.
Joanna Wiebe: How very interesting. Okay, cool. Awesome. I’m reminded, as I was watching you, I haven’t used user testing in awhile, largely because of what you pointed out, their pricing, they have messed around with that. I mostly don’t know, how do I just start doing a test? It used to be so easy, but you showed it, and it’s now easier than it looked before, so that’s fantastic. Yeah, we have some questions for you.
Ashley Greene: Okay, awesome.
Joanna Wiebe: Great, let’s dive in. Okay, let me just make sure …
Ashley Greene: I’ll just stop this …
Joanna Wiebe: I chatted over a few links already, so people can see those templates. I’ll chat them out again one last time before we wrap up here so people can follow up and get the template and get all of that, like, bonus content you were talking about, which is amazing. Susan asked the first question, do you need to refer people to a website? Could they just answer a few questions, and maybe that’s like, just answer a few questions about copy that you show them in some other way?
Ashley Greene: Yeah, absolutely. If you go back to how we would first start a test, I’m just going to share my screen again, because you’re going to tell me the questions, therefore I don’t need to look at them. When you go back to User Testing, the zoom’s a little bit in the way here. I’m just going to go back to my dashboard, and if I want to run a new test, you absolutely can. You can just upload a file.
Ashley Greene: New test, when you get back to this screen. We did it testing on a website, but you can test prototypes, a prototype URL, an Envision file, lots of different file types. You can absolutely just upload a file and get the response based on that as well.
Joanna Wiebe: That’s fantastic. Chuck asks, what are the screener questions for? I think just a quick reminder on that.
Ashley Greene: Oh, yes. Screener Questions, most of the way you choose your target audience, as you’ve noticed, is very broad. It’s like income, again, you can even choose exact income. Gender, age, things like that, but most products though are a little bit more specific. Not all products, a lot of them are more general, but the screener question allows you to say, “Well, if it’s sales product, I want to talk to sales reps,” or if it’s a copywriting product, I want to talk to copywriters.
Ashley Greene: Just to screen more specifically than the general demographic data they give you. Honestly, there’s a lot of value even for a copywriting product or something specific to do both and compare the two, because it’s kind of funny, and it really helps with clarity, but I still recommend, if you’re prioritizing, because that’s what we all have to do, is obviously start with your folks in, most of the time, unless you’re like, Wal-Mart, you’re going to have to add that screener question, and just be really picky with how you write that screener question, too.
Joanna Wiebe: That’s awesome, and UserTesting.com has become a lot better at that screener question, like the approve and reject, they didn’t have that before. That’s fantastic.
Ashley Greene: Yeah. They keep … no offense, I think they need to user test their own website because it’s very confusing. I have tried three times, knowing this tutorial was coming up, and I figured people were going to ask about pricing, because they push you to a phone call, to get on the phone. I left them three messages. They never answer the phone, to give you guys pricing information, because I have an older account and you know, I’m not … they keep changing it, I didn’t want to give anyone bad information, but you can sign up for 15 videos for $49 each. That I can tell you, because I just did it.
Joanna Wiebe: Okay, fantastic.
Ashley Greene: It’s not easy to find, but just go down to pricing, and then click “Order Now,” and it’ll sign you immediately right up, in a self serve way, even though it doesn’t actually say that. They’re really pushing you to the phone, but then they don’t answer. Still a really good product, though.
Joanna Wiebe: Yeah, it’s a very good … totally. Very useful. Alessandria says, “People who answer are selected from current visitors of the website, right?”
Ashley Greene: No, actually. You can do that. With an Enterprise account, User Testing does allow you to actually filter on your own visitors to your website. This is picking, you can sign up if you want to go and be a tester on usertesting.com. These are people who are paid to do user tests, who match your demographic screening data that you have, as well as that additional screening question or two if you put them in there. That’s kind of the basic use case, is people who aren’t visiting your website already.
Ashley Greene: Coincidentally, it may happen, but it’s pulled from a testing pool. User testing does let you test your own website, specific visitors though, but that is a feature only available in enterprise accounts and obviously there’s a higher cost associated with that.
Joanna Wiebe: Cool. Joey says, “How many responders do you use in your test group?” And this is where you said three to five.
Ashley Greene: Honestly, three to five. Even if I had the biggest budget in the entire world, I wouldn’t run a test first without three to five. If I had unlimited resources and time, I’d probably run that test, after I did the first three to five, just to see if there’s any hitches in the questions, and things like that. Again, even if you use the same questions again and again, sometimes you just need to treat them slightly for a particular group, and you know, then maybe I do ten per segment? I was trying to test, and the segment could be a different group, or just computer versus desktop, but honest to god, if you follow the three to five rule, research shows that that’s really where it starts to level off, in my experience, too. It’ll be good.
Joanna Wiebe: Yeah, awesome. Amy says, “If we have access to people who fall in our target audience, why pay for user testing rather than sending out a survey or test to those people?”
Ashley Greene: Sorry, can you repeat that question?
Joanna Wiebe: Yeah. If we have access to people who are in our target audience, why should we pay for user testing rather than just sending out a survey or a test to those people in our target audience, that we already know?
Ashley Greene: If you have software to do in house testing of your website, then that’s fantastic. I would still argue that people who know you and know your market and know your company are very different than people who don’t, have never been across your site before, right? There’s a different knowledge gap, and then you also have a different knowledge gap of just kind of the general public, versus someone who might be specific, like a copywriter, versus a female, male, between 30 and 50, average web user who makes X number of income, right?
Ashley Greene: If you have that software, great. I still say there is a lot of value, most people don’t have that software. Hardly anyone I know has that software, but there’s still a lot of value in running user tests, because there’s definitely a big knowledge gap between people who know your market, and know your product, know your company’s name and history, versus a prospect or whoever, I hate to use the word prospect. I hate that word, when they’re on your website for the first time, who may or may not know anything about you.
Ashley Greene: Surveys are fantastic, I love them, but it’s different watching someone go through your actual website and try to use it, because you’re not trying to sell anyone on a survey. You actually have to see them go through and try to do things as well. It’s really hard to do in survey form, to kind of do the exact things we were doing right now … just do both, is my answer. I’m trying to give you a succinct, amazing reason why you need to do both, but you just need to do both because you can’t accomplish the same thing.
Ashley Greene: Asking someone in a survey what they think of this feedback, versus actually watching them try to do it and suss the information out for themself, exactly like they would in real life, is what you need to do.
Joanna Wiebe: Yeah. I was thinking, with that, when you were talking about that, people who know you, there is no perfect solution here, that’s why we want to pull in data from multiple sources, right? This is triangulation at its most basic.
Ashley Greene: Love that.
Joanna Wiebe: I was thinking of this way back, like five years ago, I was working with a company called App Design Vault, and I used UserTesting.com to help me understand what to do on their homepage. One of the messages that they had was something like, save six hours by using our pre-designed templates instead of building your own, or something like that. Save six hours or something, and it was targeted at developers as an audience.
Joanna Wiebe: Now, a developer that’s close to App Design Vault would be like, oh yeah. It totally saves me at least six hours. That’s great. We saw what people who weren’t, they were so skeptical of the six hours, they were like, no, and they started doing the math. They were like, we did five tests, and two of the five developers that were looking through this were like, no, and they started calculating like, okay, it takes me this long to use Photoshop.
Joanna Wiebe: They were going through the calculations, so we quickly knew, if we’re going to do a save time claim for this particular audience, we’d be wise to actually break down how much time, do the calculation for them. Maybe even have a calculator so they can see how much time they’ll actually save, where they can enter it and feel really confident in that, but if we’d asked only people who already knew App Design Vault, we might not have had the same insights into that skepticism, with that fake message.
Ashley Greene: That’s a fantastic example. People who know you versus people who don’t, there can be a dramatic difference, right?
Joanna Wiebe: So do both.
Ashley Greene: Do both, exactly. I really recommend, we chose the copywriter audience specifically here, but do both. I would do it with your own website users, your own people already using your website on here with your target group, and a more general one with just [general graphic 00:38:55] data, and triangulate just like Joanna said. Three to five with each. It’s still 15 videos.
Joanna Wiebe: Yeah, love it.
Ashley Greene: … and pattern recognition. Really doesn’t take that long to do the analysis, either. If you have three videos, ten minutes each, 20 minutes each of analysis, we’ve got 60 minutes of analysis and insights. You may find you have to do a couple more videos, and you go and do a couple more videos, but often, you’ll see some patterns almost every single time, in three to five.
Joanna Wiebe: Yeah, love it. Chuck asked, can UserTesting.com be used with longer sales letters and landing pages?
Ashley Greene: Absolutely. Any URL, website you want to throw at www.UserTesting.com, you can upload it. It’s great for that. Sign up flows, checkup flows, you know, going through a sales, anything you want. If it has a URL, you can put it into UserTesting.com.
Joanna Wiebe: Awesome. Alessandria says, if your website is a very specific industrial B2B company, would this work?
Ashley Greene: Yes.
Joanna Wiebe: Just that, answer, yes.
Ashley Greene: Yes. No, I have a tendency to go on and on, so no. Yes, again, there are a couple circumstances where maybe they can’t recruit the exact right testers for you with that screening question. That being said, I’m surprised at how many niche audiences they can really get. I’ll be surprised. If you follow to that and they can’t get anyone in your audience, what I would tell you to try first, asking people who fit the demographic profile, the thing is with bad messaging, clarity, confusion, you’re never, even if you’re targeting a more specific audience, you’re never going to go wrong by having more clarity. Again, that very general, honest, can provide clarity, even if they know absolutely nothing about your industry.
Ashley Greene: Then you need to be able to understand what your words say, mean what you think they mean, and a bare minimum, and I’m just going to the absolutely bare minimum to convince you, that is invaluable.
Joanna Wiebe: Yeah. Okay, Jessica says, you’re using this for a homepage, but can you also use it for email marketing messages?
Ashley Greene: Absolutely. Basically, there’s … the easiest way to do that is to take the email and just get the web URL for the email, and then you can throw it in as testing a website, or it can do it as testing a prototype. Generally, as long as you have a web based URL you can just throw it under testing a website. As long as people can access it, it’s fine, and if you don’t, don’t have a web URL yet, just mock it up in a document and upload it as a prototype, and in part of the opening instructions, just say “This will be an email that would be sent out, and you would receive it in your email box.”
Ashley Greene: Just give them that context, in that opening statement where I said, you know, please read the questions out load, we’re going to be asking questions about a home page, in that circumstance you would just get a little bit more specific context that it was an email, you would normally get it in your inbox, blah, blah, blah.
Joanna Wiebe: Cool.
Ashley Greene: But absolutely.
Joanna Wiebe: Yeah, I think it would be really interesting. Jennifer says, “Is it possible to test small blocks of copy that haven’t gone live yet?”
Ashley Greene: Absolutely, all you need to do is upload a file. It’s pretty flexible as long as you have a file. Honestly, there’s another trick … and I should have had this prepared, there’s, and I can probably include this in the bonus, or Joanna, you can send it out with the email. There’s a really quick way to take any website you want, you just type something in here, and you can change all the copy on the page. Literally instantaneously, say if it’s an existing web page, you could just go through, change the copy. Screenshot it, upload that as a file, so that way it’s already in there, if you have no innovation skills, or prototyping or wire framing skills or anything like that, you can google that, too. I’m sure you can find it.
Ashley Greene: It’s very well known across the internet, and I should have had that ready for you. It’s just a little bit of HTML that you plug in. It takes five seconds, but even if not, bare bones, upload a basic document, give people the context, and you can test it that way.
Joanna Wiebe: Love it. Yeah, and there are chrome extensions galore for that. That’s awesome.
Ashley Greene: There we go, that’s even easier.
Joanna Wiebe: Allison says, “At what stages of a page’s life cycle do you most recommend user testing?”
Ashley Greene: A page’s life cycle. Any … yeah. I think I … Joanna, you tell me what you think it is.
Joanna Wiebe: I think it’s in the workflow. Allison, if you want to chat over what you mean, like first draft, or just when you’re about to go live or after you’ve gone live, what is that, Allison, if you’re still on because … well, you chatted this about ten minutes ago. You might still be around. Yeah, I totally mean in the workflow. Yeah.
Ashley Greene: Okay. Again, there’s never too much thing as too much user testing, but I get that’s not practical advice, right, because there’s only so much time and you have to do it when you’re going to get the biggest bang for your buck. I would do this once you’ve done your research, and you’ve drawn up your messaging, and you’re feeling pretty good about it. I would definitely do it at that point. If you’re having any internal circles around, I would definitely do it at that point.
Ashley Greene: If you’re a copywriter and you’re doing it for clients, before you send it to clients. It’s more at the later stages. It’s really about refining versus creating for the first time. Not to say you don’t find insights actually change your messaging completely. You can, and you will, but the really, best use case for user testing for messaging, in my experience, is to use it to refine what you already have, and kill bad messaging fast.
Joanna Wiebe: Okay, nice. Okay, Richard says, “When selecting three viewers, is that deemed as three videos, or is that just one video?”
Ashley Greene: When you do at the top, when you go to create a test, just going to go test a website, this will help us, so five participants. Right here, this is the number of videos you’re going to get. You’re going to get one video per participant.
Joanna Wiebe: Okay.
Ashley Greene: Then you’re charged at your … you know, $49 rate or whatever rate you get from UserTesting.com, but this will be the number of videos you get.
Joanna Wiebe: Okay, perfect. Just for some other chats going on here still about the “Edit this website” thing, the extension I use the-
Ashley Greene: Thank you.
Joanna Wiebe: … edit, it’s called. Sorry?
Ashley Greene: No, I was going to say thank you because I just have that HTML code that I use and I should have had it out and ready, but I probably couldn’t find it in ten seconds, so.
Joanna Wiebe: Yeah, no, that’s all good. I’m just going to chat it over, because someone just said, “Which do you use?” It’s called Page Edit, and it’s a Chrome Extension, and it works perfectly fine, in my experience, for editing a page.
Ashley Greene: Awesome. I’ll have to use that, too. That makes my life easier, thank you.
Joanna Wiebe: Yeah, it’s actually really helpful.
Ashley Greene: Yeah, no, it’s awesome.
Joanna Wiebe: If you were, this is anonymous’ question, “If you were to use UserTesting.com for email, would you still use the five second test? Would that still be something that you would use as a question?”
Ashley Greene: Interesting. I haven’t had that question.
Joanna Wiebe: Yeah.
Ashley Greene: If we go back to, let me just review. I’m going to just quickly review the five second test questions, just … well, honestly, I really love the five second test, because with an email, we all know that, particularly optimizing for subject line and open rate, you’re going to get the skim. I would still do the five second test with regards to the subject line or what they can see. You know, what you would expect them to see if they opened it up on their phone, you know, but really, the subject line, I still think it would be worth it. I can’t say I’ve personally done that, that’s never been … I haven’t done a whole lot of email testing, I’ll be honest, but I love that test, and it’s really good to diagnose what’s confusing. What about you, have you ever done it with email?
Joanna Wiebe: You know, I’m thinking the way … I haven’t done it with email, but what, because it just wasn’t something that I was doing, but-
Ashley Greene: Yeah, it’s a great question.
Joanna Wiebe: It’s interesting. Let me share this. If you were to write, what I would I would imagine you would do, and so it’s worth anybody just trying to do it their way, but what I imagine is you’d want to write it in whatever tool you’re using, and do a series of thoughts, likely, and put those in a Google Doc, and I would start that with, and I’ll just chat this out.
Joanna Wiebe: Exit Intelligence has this really simple, free tool, where you type your subject line in, and the opening lines of your body copy for that email, and then you can see it in a phone, what link would even go there.
Ashley Greene: I love it.
Joanna Wiebe: Take a screen shot of that, and then you could ask with the five second test, what do they remember? If they don’t remember your subject line, amid six other subject lines, then you might have a problem on your hands. It doesn’t mean you absolutely do, but it could be a good thing to get a sense for, is my subject line even noticeable in the inbox? That’s probably where I would want to start, but that doesn’t mean I have before.
Ashley Greene: Yeah. No, I mean I think it’s a great question, and I think that’s a great way to do it. I think it would be really helpful again. Where it’s gonna be helpful is exactly what they’re going to see from their phone or their computer when they decide whether or not to open up your email, and it can be really helpful, I think, for that, but I have not personally done it. Now I’m going to have to go try it, because now I curious.
Joanna Wiebe: I know, I know.
Ashley Greene: I like how Joanna put that out there. Yeah, subject line I would definitely test it, and let us know your results, if you do.
Joanna Wiebe: Yeah, absolutely. Okay, I know we’ve used up so much of your time. There are three more questions.
Ashley Greene: Yeah, no, that’s okay. I’m good for another eight minutes.
Joanna Wiebe: Okay, good. I’ve chatted out the bonus content page, so be sure to go there, guys, the question template as well. Ashley, I do want you to take a quick second to talk, though, about before we answer these final questions, because I have a Connect with Ashley Calendly link, can you talk to people about what’s going on there? Why would I share that?
Ashley Greene: Yeah, and absolutely, thank you. I am, my company is Stratify, we do customer insights, and my whole shtick is customer research, quantitative tells you the what, qual tells you the why, and a lot of people over rotate it to quant, but we know, especially the copywriters here, and hopefully a lot of startup marketers and growth people know the value of qualitative. I’m launching a course based on all the teaching I’ve done at accelerators and things like that, and taking a very product development type of approach, which is, I’m beta-ing it, and I’m really handpicking, you know, I already have the majority of people. Handpicking some people to go through it with me. Obviously, there’s great early bird pricing, but I really want your feedback.
Ashley Greene: If you’re interested, I’m going to tell you the five modules of what we’re doing. Just sign up for a quick call, and I’ll guarantee a walk away of something actionable no matter what, just to assess your fit and to see if you’re interested and give you a little bit more information. Again, we all know about betas, the better more hand holding you do in the beginning, do things that don’t scale, that’s what I’m doing.
Ashley Greene: From a product development standpoint, and my own research, my personal experience, but that’s anecdotal, we want to base it on our customer research. I need a plug, I need to give you guys a plug and play system to get a job done. I’m not going to teach you how to do a customer interview, or how to do a user test, you know, protocols and going to unbiased questioning, and all of those things. It’s going to be like we did today. Here’s a specific job to be done. I’m going to walk you through exactly how to do it, and give you those kind of insights.
Ashley Greene: In the five kind of modules we’re going to be doing, one is pricing products to move, increasing conversion rates on landing pages, websites and emails, new product developments and developing a new digital product people will love. Finding new markets that’s super fans, if you’re going into a new market, and then lastly, finding those untapped, sustainable acquisition channels, and if you find one, obviously that’s amazing, and you’d be surprised with doing a little bit of dirty work, quick and dirty qualitative research how you can find those. There was one in the wedding space, they figured out that all had something with Starbucks and in common, and just using that one thing, opened up so much lower cost for conversion on their paid, and just for [inaudible 00:51:14] target that way, and with some organic content.
Ashley Greene: That’s just like a superficial example. A lot of examples, you’ll find partnerships and actually some untapped channels your competitors aren’t on, because they won’t do the hour or two it takes to do that work.
Joanna Wiebe: Yeah. Amazing. If they connect with you at that Calendly link, then you’ll see if it’s a good fit for them to be a beta user.
Ashley Greene: Absolutely. You know what, you can email me at any time, love to hear feedback, any questions you have, at ashley@instratify.co. I did have a page in here, apparently it disappeared on me, with my email. Feel free, if you’re ready just to hop on and schedule a call, just 15 minutes, we can chat and see if you could be a good fit for what we’re doing, the Calendly link would be great, and then any feedback or questions, don’t hesitate to email me or tweet at me for anything we covered here today.
Joanna Wiebe: Perfect, okay. Those links are shared out, thank you for that, Ashley. We do have these final three questions.
Ashley Greene: Yes.
Joanna Wiebe: We’d like to [inaudible 00:52:17] the next week.
Ashley Greene: We’ll do it. I’ll be succinct. Something I need to work on.
Joanna Wiebe: I’m bad at that, so good. Simon says, “Could you use user testing to do a google search for a specific key word and comment on the impact of your adwords ad versus other ads and natural search results?”
Ashley Greene: Absolutely. Any website in the world, you can do. You can literally have your test, you have them start on Google, and then get to your product. You can have them go to a URL in the middle of the test. Absolutely. If you look, I think maybe even listed out here. They have some really great examples in here. They had one here … here, for Amazon. Let’s see, I think this is a great example, just kind of quickly being succinct, show you, so you could basically have them start on Google, have them google keyword, which you should have ad pop up with, have them follow it through your landing page and your flow. Absolutely. If it’s a website, you can definitely go to multiple websites during my session. During a single session, we can have them going to start in google, then get to your page or go through your flow.
Ashley Greene: Honestly, everything we did today, here’s the biggest, one of the biggest tips that gets everyone super excited. Run it on your competitor’s websites. Even if you take exactly the questions we have, and just modified them and ran them on a competitor’s websites for messaging insights, that would be great. There’s a specific set of questions, I do change some for competitor research, but even if you just take the questions we have today, modify them slightly and ran them on your competitor’s website, any website in the world, you can do.
Ashley Greene: You’ll love it, and if you have clients or a boss, they’ll love it because everyone likes to know what their competitors are up to, and how you compare.
Joanna Wiebe: Yes, I love it.
Ashley Greene: Whether or not that’s important is another story, but we all know, job to be done, we’ve all got to do it. You know.
Joanna Wiebe: Cool. Mark has a question that I can answer quickly, and the start of this, will be recorded and made available to Copy School members where can I find it in previous tutorials. You do not have to be in Copy School to get this replay. It is, if you go to CopyHackers.com you can look in the top bar and in the sidebar, there are links galore to these tutorials for Tutorial Tuesdays, including this one. That will be, Today’s will be the most recent one showing, or the first one that you see when you go to CopyHackers.com and look at those tutorials, so that is that, and that brings us to the final question. [Bikah 00:54:40], I hope I’m saying it right, says, “Do they make user testing in other languages, like French?”
Ashley Greene: That’s a really good question. I believe so. I believe you can ask, I have never had to do this, but absolutely, I can’t see a reason why not, you can choose your test region, and you can put your questions in that language. If you look through our target market assessment, and I think that might also be a feature available in the Enterprise plan, because they have a lot of features in that. You can give them a shopping cart and they can actually go through and buy a whole product, so you can really check out out your sign up. There’s so many things you can do, but even with this basic plan, I believe you could just pick your country, no, okay. Okay, it is English speaking for this.
Ashley Greene: I do think in their Enterprise plans, you can pick more countries and can do it based on languages, but you could. Here’s the hack, I haven’t tried it, you know, go to the best of the world, and then in your screener question, ask for people who live in that country or speak that language, and then have your questions be in that language. If they can’t answer them, they can’t answer them. And again, any URL in the world, you can use. It shouldn’t be a problem. You can try it that way.
Joanna Wiebe: Fantastic. Ashley, this has been super duper valuable. Thank you so much for having people chat over and they have been chatting throughout, I know we had to, for a 20 minute tutorial we went for an hour, but that happens all the time.
Ashley Greene: Good.
Joanna Wiebe: Some people will have already had to leave, but we have the replay available, people can see it afterward with the transcript as well. Plenty to follow, and I’ve chatted out some final links again, that bonus content and the question template, how to connect with you, if people want to take this training further, and then of course, your twitter handle, too. Awesome, great stuff, thank you so much, Ashley, amazing.
Ashley Greene: Thank you for having me, it was really fun.
Joanna Wiebe: Cool, I thought so too, and everybody was very pleased with it. Thanks everybody for participating, for your great chats and questions along the way, and for sticking around, too, and I’ll see you in our next Tutorial Tuesday. Thanks Guys!
Ashley Greene: We get to take a peek at your office.
Joanna Wiebe: Oh yeah, right.
Ashley Greene: I’m super excited for this one, because I doing a new office too, so I can’t wait.
Joanna Wiebe: Fantastic. Okay, cool. Thanks guys, see you then, buy.
Ashley Greene: Yeah, bye.
