Episode 10: “The Importance of Research”
Good research is essential to anyone wanting to excel in any business, and in particular any online business. We break down how and why we do all of our research for our projects.
Summary
Nick and Kai walk through how research anchors their respective practices, UX design and outreach/PR, and why they both treat it as a non-negotiable condition of any engagement. The conversation covers practical methodology (passive interview recruitment, competitor backlink analysis, survey design) and broadens into a critique of Silicon Valley’s move-fast culture, with both arguing that small, deliberate businesses outperform VC-funded sprints for most people.
Highlights
- Kai’s outreach placement rates run 25–35% against an industry average of 8–12%, a gap he attributes to front-loading research on both target site selection and pitch language, matching how a specific audience actually talks to each other.
- Nick uses ethn.io to passively recruit interview candidates via a 3–4 question disqualifier survey on the client’s site, offering a $50 Amazon gift card. Qualified candidates book an hour-long call through Calendly; Nick goes in with 10–11 prepared questions and spends most of the call silent.
- Nick’s A/B test success rate is above 50%. He ties this directly to researching what customers need before writing tests, rather than guessing at headlines and call-to-action copy.
- Slack determined internally that the only metric predicting paid plan conversion is 5,000 cumulative messages sent across the team, not user count, not integrations. Nick uses this to illustrate why acting on the wrong metric wastes time regardless of how much data you have.
- For customer surveys, Nick uses even-numbered response scales to force a real opinion (a 1–5 scale produces a pile of threes, which tells you nothing), cuts any leading questions before fielding, and points to Pew Research polls as a model for neutral phrasing.
- Kai’s tactic for capturing customer language: read 1-star and 2-star Amazon reviews of top-selling books in a niche. The exact phrases reviewers use to describe what the book failed to solve are the problems your audience wants addressed. Use that language verbatim in copy, headlines, and subject lines.
- Both treat research as a condition of the engagement, not a line item to negotiate away. Kai tells clients upfront that month one is research only. Nick has started refusing work that does not include a significant research component, and says he leaves if a client tries to cut it.
Read the transcript
Everyone I know who is running a good, solid, durable, bootstrapped business moves pretty slowly and carefully about stuff because they can’t afford not to. They’re not given $18 billion to conquer the taxicab industry. And neither are you. So Kai, how does research fit into your business? What do you do with research?
Well, Nick, I found that when I started out doing SEO and outreach, I discounted research as being A necessary part of the process. I thought it would be like 10% figure out what we’re going to do, 90% execute. But after a few campaigns, I found that Really, it’s 90% research to understand the strategy we want to use, the specific tactics we want to use, the implementation we want to use, and then it’s 10% executing on that. In the nitty-gritty, research for me is working with a client to understand the goals of the campaign, what we’re trying to promote, who we’re trying to reach, who our ideal customer is. Then going out and validating that. So the client says, like, we’re trying to sell, you know, blue widgets, and we want to sell it to women aged 29 to 35. And We think like these types of communities will be the best ones to reach them. So I go out, I look at those communities, I look at that type of persona. I see, are they buying it? Are they interested in it? Are they asking questions about it? Are they sharing competitors’ products like that? And I gather up that research and use it to form the foundation of the strategy for the engagement. So for me, it really goes through a process of discussion with the client to get a dump of what they know. pass it through my filter, go out and validate it, make sure we’re on the same page and in the right direction. And then from that, drop a short strategic brief that says like, hey, based on the research you shared with me, based on the research I did, this is the direction that makes the most sense for us to go. Occasionally, I’ll offer a choice of yeses. We could say we could do option one, option two, or option three. I heavily recommend option one. In other cases, I’ll say options one is the best. Let’s do option one for three months, and then circle back and decide how we want to move forward from there. How about yourself?
So I’m a UX and interaction designer by trade. And so research means everything really for us. You can’t have a good UX project without a good research process. I firmly believe that. It usually involves a combination of qualitative data, which I define as just going out and actually talking to people and listening, which is hard. We’ll get into why that is hard. And quantitative data, like heat maps, Google Analytics, sales data, that sort of stuff. I believe that both of these are important for very different reasons. Quantitative data is good for getting rid of like kind of the biggest, like Detail issues. Oh, is there a default text that’s tripping people up on this form and holding them back from converting? Let’s change it. You can find that out very, very well with quantitative research. Is there some copy that’s tripping people up or making them bored and keeping them from scrolling further down the page? Get rid of it or move it or reword it. and see what happens. It’s tactic stuff. The qualitative research informs the strategy that you’re going to be putting together. And that is just tremendously important. You really don’t know why people buy until you talk to them. You can survey them. That’s a good form of qualitative research. Sometimes you get some quantitative insights, like a net promoter score or something like that. But I my favorite type of research involves passively recruiting people. So I’ll throw something like ethneo. If you go to ethn. io, it’s a widget, like a JavaScript widget that you throw on your website. that puts up a survey. And the survey is meant to disqualify you as a potential candidate for a call. So have you ever seen this product before? How long have you used it? What’s your age? Where are you from? What language do you speak? Stuff like that. Just little things. And you want to make it like a three to four question survey, and you say, Oh, do you want a $50 gift card from Amazon? Take this survey, and we’ll call you back and give you a brief interview, and then we’ll give you a $50 gift card from Amazon. I will basically, if you’ve succeeded, I will say, hey, We’ve chosen you to participate in an hour-long research call where we’re just going to ask you about your experience with this site. We have your phone number on file for Methneo, and just set up a Calendly appointment with me here. If you go to Calendly. com, I use that to book out my schedule. And we’ll be back to you at the point of time. And I’ll have my assistant follow up the day before and say, hey, You know, you’re going to have this call tomorrow. Please stay in a quiet place. Make sure that you have the time for it. Please let us know if you can’t make it, etc. And then I get on a call and I talk to them. I have maybe 10 or 11 prepared questions. And most of the time, I am shutting up and listening. It is very, very hard as a practice to shut up and listen. It is something that you ha I’m still only barely halfway good at. You are trying to reflect their statements back to them and get out more insights from them. Pausing. There are a lot of times I leave awkward silences on phone calls so that I can get more information out of the person. and figuring out how to ask questions in a natural manner, trying to empathize with their situation without offering suggestions. So if they got tripped up at a given page Do not assume it was their fault. It is usually your fault if they got tripped up on the page. Besides, they’re the one with the money and it was hard for them to part with it. So maybe you should try and fix that. Qualitative research is something that really it humanizes your customers, it gets people invested in the narrative, and it Allows them to understand a lot of strategic changes that you end up doing with your product. I’ve done this with numerous clients over the past nine years, and I’ve never Had a client come back and say anything less than, oh my God, this was completely amazing and it changed my view of how we are actually running business. Full stop. I’ve never had anybody say, Like, oh, this wasn’t worth our money. Nobody has a lukewarm response to research. No one. Research is the bedrock that guides the design process. You cannot have a design process that is based on intuition or speculation. It should be based on hard data. And that data can be people, that data can be numbers. you’re still gathering information and acting on it. Otherwise, you’re flying blind. Too many people fly blind in the tech industry, and it sickens me.
No, and I love that last point. And jumping back to my research methodology, I think I hit on the qualitative points primarily before quantitatively, specifically in an outreach campaign. Like if I’m working with a client to get them on podcasts or working with a client who has an e-commerce product to get them reviews, there’s a strong quantitative component where And my exact methodology is: who are our top four competitors? Okay, let’s go to our competitors’ websites and first let’s go to Google and just search competitor name review and see who’s reviewed a competitors’ product before. Then let’s go to our competitors’ website and use a backlink analyzer tool like Majestic SEO and just see who’s linked to our competitor’s website. Since 90% of the time, when somebody writes a review or somebody’s on a podcast, it links back. And so then we just say, okay, there’s 200 people linking to the site. Let’s go look, see it, qualify these sites, throw out sites that are too large, sites that are too small. Leave just the sites that are just right, and then manually screen through them and say, well, we’ve quantitatively gathered up potential leads. Qualitatively, let’s look at them and figure out who feels like a good match. And then from there, we’ve built a list of people to reach out to. And that really becomes an integral part of the research process for an outreach campaign. Who are we trying to reach? Who specifically are those people? And how do we qualify a list of the best buyers or the best prospects to reach out to? Yeah, yeah.
This even happens in my A-B testing engagements. I run heat maps on all of my websites. I have been starting to ask customers more often and actually get surveys from them. I’ve really tried to listen to them more often. I’ve been diving into Google Analytics more. I’ve been trying to act On these things, and I’ve always found that when I do that, I end up with better A-B tests that succeed more frequently. And you aren’t going into doing an A-B testing strategy so that you have a one-fifth success rate. My success rate is over half at this point because I listen and I try and figure out what people actually need. And it’s different for every website. There’s no one s secret success that works for everybody. I used to fly blind like anybody else, like changing call to action buttons and headlines and stuff like that. And I still do that sometimes, but I do it because I know what the messaging should be. And that allows me to stay a little bit more informed about it and say, okay, this is a better hunch. we can do this headline this time.
Oh, just touching to that, like you’re saying, the more research you do, this has increased your success rate on A-B tests over that 50% threshold in the outreach world. An average placement rate. So reaching out to a site, them saying, yeah, we’ll link to you, or yeah, we’ll write a review, or yeah, we’ll have you on the podcast, might be somewhere between 8% and 12%. I’ve been able to consistently get placement rates of 25 to 35 percent, if not more, by investing heavily in research, both by identifying the sites we want to reach out to and screening out the bad fits. eliminating a bad test essentially, or researching the messaging we want to use when we reach out to these people. How do they talk to each other? What language do they use? What words do they use? Do they describe it as a giveaway? Do they describe it as a review? Do they describe it as a product review? Do they describe it as an interview? What Messaging, do I want to use to have it resonate with them in a better way? And if I’m suddenly speaking their language, similar to, I’m sure, in A-B testing, like we’re optimizing towards speaking the language the customer wants to hear and expects to hear to move them forward in the funnel. Same with outreach. I’m optimizing the language I use in my outreach emails to speak to the user, make them feel more comfortable, and move them forward in my outreach funnel to get to the conversion point of Yes, we want to have that on a podcast. Yes, let’s book a time. Yes, send us a product for review.
Research has made me appreciate A-B testing again. I started to get a little burned out on it, and now I feel like I’m. developing a methodology that resonates with my values as a designer better because it meets the user and the reader halfway. It’s all about figuring out what somebody wants and trying to meet those needs and address that in a way that Don’t make me want to pucker my butthole, you know? It’s not slimy or horrible. And I mean I think that both of us would agree that research has kind of formed the most important aspect of both of our processes. One question I wanted to ask you, clients, a lot of prospective clients I get. They look at my process and they’re like, okay, well, you’re going to do research, and then it’s going to result in IA and sitemap and user flows and a data model. And then after the data model, we’re going to run into key pages. And then after Key pages, we’re going to make detailed wireframes. And after the detailed wireframes, we’re going to have a periodic check-in to make sure that we’re building the product correctly. Like, great. Can we get rid of the research? I don’t know if we need the research. How many times do you get this conversation? Do you get this?
I get it a decent bit. I’m a hard ass and. Uh, early on in the process in my welcome packet, in the initial conversations, I flat up tell people: like, month one of our engagement, we’re not doing any outreach, month one of our engagement, I’m Researching your business, researching your customers, researching what the next steps need to be. Because how the hell can I operate if I don’t understand your product or the people we’re pitching to? Research has to be the bedrock, has to be the fundamental that we start from, or else we’re going to be shooting in the dark. And so, whenever a client tries to cut research out of the process, it’s either A go nuclear event where I’m saying that it’s absolutely necessary. This is why it’s necessary. Or it’s a fireable offense where I say, like, we’re not going to be a good fit if you think I could just jump in and start pitching a product to people. It’s not going to work that way. I can’t magically know your product or know your audience well enough to pitch it effectively. I could pitch it half-assidly and we could get a 5% placement rate, and we’ll all be disappointed. Or you can invest a month in research and 4x, 5x the response we’re going to get. Which would you prefer, dear client?
Agreed. Agreed. I have begun refusing work that does not include research as a significant component. And you have to be okay with us taking the time to do it right. I know you want to move fast and break things. I know you live in an industry that incentivizes that. But You are going to make the wrong decision and leave a lot of money on the table if you don’t invest upfront in research. Because others are. Your competitors are. And if they aren’t, they will because they’ll hire me.
I once got on a call with a client who used that exact phrase. You know, we’re a startup, we’re in San Francisco, we really like to move fast and break things. And I said, I don’t. I move slow, I move deliberate, I’m lazy. It’s going to take a while, and I’m proud of the fact that I move slow. And yeah, we’ll get to it when we get to it. Are we still going to be able to work together? And they said yes. And it ended up being a good engagement. But like, I just took the bull by the horns. They were like, we want to hustle. We want to move fast. We want to get this going. And I’m like, no. No.
And when I was when I was in Hawaii, I was on this one beach on the big island, and I saw like 15 sea turtles on this beach, and they must have been like 85 years old. And I watched them for like a good hour. This was my vacation. I just hung out and watched sea turtles. And during that hour, the biggest one of them, who looked clearly the oldest, he moved once. And I just think, like, I want to be like that guy, man. I want to just sit there all day and like think a lot and then be like, I’m going to go here now. And then I do, and that’s it. It’s measure twice, cut once, measure 18 times and cut once. That’s why you should be hiring a consultant. It’s for their thinking and their methodology, not because they’re magic. It feels very counterintuitive because I’m extremely fast at creating wireframes. Like, I can make an 80-page wireframe deck in about a week at this point. And everybody on the call is like, whoa, that’s bullshit. No, I can do that. I’m very fast in Omni-Gravel. But like that doesn’t matter. I could make the wrong thing. You know, I have made the wrong thing and it’s horrible I would much rather do that research. There are so many people that support research that are far smarter than me. Erica Hall wrote a book called Just Enough Research that is fantastic. And she says, Just what you said. It is a gonuclear provision in my clients. If they try and cut research, I leave. Jared Spool talks about the value of research all the time. His agency is entirely oriented around good research. Who else is smarter than me and does this? Tomer Sharon. I don’t know if I’m botching his name, but he is the chief of UX at WeWork. WeWork is moving fast. But they’re not breaking a whole lot.
Yeah, there’s.
They’re breaking some things, but you know.
There’s an unfortunate. There’s an unfortunate acceptance of like the, I don’t know what to use. That’s not a disparaging anti-Silicon Valley phrase here, but like the hustle hard, move fast and break things. I don’t know, sure, maybe. Like, I think back to my newspaper in college, and we moved fast. We offended people, we shipped as fast as we could, but We also were like very deliberate and strategic in it. And we had a plan. It wasn’t like, we got to shove content in here and get it out. It was like, cool, we’ve got a four-week editorial runway. We’re making sure things are happening. We’re working hard. We’re moving fast. Figuring things out as we go along, but we have a plan here, and we’re moving forward with the plan.
Yeah, yeah. Being there is, I knew this was going to happen. This episode began with research and is turning into a disquisition on being thoughtful and deliberate. And it happens in your business, and it happens in your actual work. It’s something that is extremely near and dear to my heart. Like I the narrative that the Valley feeds you about the way the tech industry moves is Warped and fucked up and horrible, full stop. And everyone I know who is running a good, solid, durable, bootstrapped business moves pretty slowly and carefully about stuff because they can’t afford not to. They’re not given $18 billion to conquer the taxicab industry. And neither are you.
And it’s amazing how apologetic people could get about the fact that they’re choosing to run a small, deliberate business. I had a coaching call with a student a few, about a month and a half ago, and we were talking about her business goals. On the call, she spent about 15 minutes saying, I feel guilty. Like, I just want a small business. I don’t want to build an agency. I just want it to be me. I want to have it grow and stay solo. And I’m like, that sounds good. What’s the problem with that? I feel like, like, people are telling me I need to build an agency and I need a team, and like, how do I grow otherwise? And I’m like, raise your rates, do good work. Make sure you’re keeping like 10 hours a week of that 40-hour week for you and keep on doing it that way. It’s working for me, it’s working for Nick, it’s working for our friends. It’s the idea of building a small business. The only thing is, our small businesses happen to be online. Instead of a storefront or like hanging out a shingle for our design agency or our PR agency, we’ve got a website, and people show up and They engage with us, and instead of a radio show, we have a podcast. But it’s that whole idea of like having a small, durable business instead of Hey, I’m going to get some VC and go for the moon and see what happens. Patrick McKenzie had this wonderful, wonderful quip that I always butcher. Winning, IPOing, winning in the San Francisco gold rush, in the VC gold rush, is like rolling a 100-sided die and waiting for a 101 to come up. And, like, it can’t happen. That’s not on the die. Like, you’re asking for an impossible outcome. How do you sound? It’s just to land on the edge. Yeah, exactly. Like, flip the coin and it lands exactly on the edge. Do that 10 times and have it land on the edge 10 times. Congratulations, you could IPO now. Like, it’s it’s there’s a reason they’re referred to as unicorns, they’re biblical. It’s not really going to happen for everybody. You’ll be more happy and successful building something small and durable.
And you know what? I read like an insider publication about the Valley stuff, and employees at those companies are getting fucked on private stock right now. So even if you’re like the director of product at a successful VC funded startup, great. That’s your pedigree. You’re not making much. And the way the market is going right now, you’re really not making much. All of the investors are taking all the money from all the founders, and you’re basically screwed. Enjoy being screwed. Even if you’re at a unicorn, you’re screwed. Figure out what you did wrong and come back to Chicago where we actually build real businesses. Thanks.
Jumping back to the idea of research for a second, I’m curious: how do you iterate on research? So, you’ve gone through the research process once, you have some data, you’ve started moving on it. What do you do next? How do you iterate on that and take that research to the next level?
Well, so it’s a process you create the research, you act on the research, and then you determine if you were right or not, right? Usually what I do is I make research, I act on research, and then I research again. That research usually takes the form of analytics and more quantitative sides of things. It’s hard to compare qualitative stuff against other qualitative stuff. But I really like it, it’s a never-ending cycle. You should be researching and creating and researching and creating and researching and creating until you go out of business. Or retire. Or die.
Preferably not the third one, dear listener.
Well, we all die, you know, it’s fine. it’s just a continual process that you have to be following. And so, you know, I look and say, okay, well, was the research right? Is the research not right? Usually it’s not entirely right. What did I get wrong? How did I get it wrong? Was it that I was acting on the wrong data? Was it because I was collecting the wrong data? Was it because I was coll Collecting too much data Was it was because I was collecting data that I shouldn’t have acted on, and I should be acting on this other data over here. So for instance, Slack, the messaging app. They have a metric that they have determined in-house determines the success and likelihood of you upgrading to a paid plan. And that is, you have sent Cumulatively, across your entire team, 5,000 messages. That’s it. Doesn’t have anything to do with the user count. It doesn’t have anything to do with how many integrations you have. And it doesn’t have anything to do with how many animated gifs I’ve pasted into your Slack. It entirely has to do with whether you, as a society, on this Slack, have sent 5,000 messages, and that’s the only thing that they care about. Revenue directly maps to it.
That’s amazing.
And so I can easily imagine a scenario where somebody’s going in and talking about monthly active users. Fuck your monthly active users. It doesn’t matter. You can have 18 million active users among 200 businesses in Slack and not pay them for it. They can all be on the free plan. They could easily be in a situation where they make zero dollars. I’ve been on Slacks that are 20,000 users and they’re on the free plan. What? Yes. I mean, I immediately leave. Because General just goes all the way up the screen and like. And I’m like, oh my God, my browser caused my fans to turn on on my laptop. Like, I need to nope out of this.
It’s very similar for me on iteration. So, iteration. Iteration partly occurs on the audience front. Which audience are we targeting? Who do we want to reach? What podcasts are we trying to reach? What’s our messaging and our pitch? I iterate on that a bit, but more often what I’m iterating on is the outreach emails themselves. What emails am I sending to these people to open a relationship, build that relationship, and then turn that relationship into a placement or an opportunity or a joint venture? And I find That’s the area where iteration makes the most sense, since that’s where I have the most, I guess that maps most to like Interaction within an A-B test. That’s where people are opening the email, clicking on the email, responding to the email, doing some action that I’m asking them to take. And That’s where I end up iterating the most. How do I improve this email? How do I make it speak the customer’s language better? How do I make it have a clearer call to action? Move the person forward in the process to get them to respond. I’ve had some campaigns that are really successful off the bat and get a 75% response rate. I’ve had some using the exact same templates to a different audience that get a 20% response rate. And Well, great. Now I have to say, like, well, what’s different about these two things? How do I make the new campaign better match the audience I’m trying to reach to or trying to reach? Building on the best practices that I’ve established. I’m a big fan of saying, like, there are best and better practices that you learn over time that you should incorporate to campaigns, whatever the type of campaign is across whatever industry. But within that, you have to be aware of when your better practice is in opposition to the audience you’re trying to reach and say, well, this works 90% of the time. This is the 10% of the time where it doesn’t work. Let’s throw it aside and try something new. We have to throw away research, old research, or new research, or research for a campaign. to be able to see clearly, to throw away the assumptions we have and say, well, what don’t I know here? What don’t I know that’s causing this to not succeed as I expected it to? And how do I learn what I need to know to make it start succeeding?
Yeah, yeah, exactly. I mean, one common thread you, dear listener, might have figured out over the past ten episodes of Make Money Online is that we screw up constantly. But we know how to recover from it, right? Like, we know how to say, okay, well, that didn’t work. This is awkward. Okay, let’s try this thing again. And you’re doing this while sounding authoritative and trying to, you know. I don’t want to say save face, but you’re definitely doing what you can to say, okay, well, you know, this won’t work. Or it only works a certain portion of the time. Set expectations ahead of time in those situations. I’ll always say, you know, it’s UX. Something where I have a hunch that it’s going to work because the process has worked quite a bit in the past, but there’s always a possibility it won’t work. Here’s what happens if it doesn’t work. And if you get ahead of it, then it’s a lot easier than being caught flat-footed, you know?
Absolutely. Absolutely. So one question I wanted to ask you, this is something that I’m starting to work on, and I’m really interested in your answer. What steps do you take to make a good customer survey?
That could be a whole episode, man. Part two. Stay tuned, everybody. So, good customer survey. You. Number one thing. Write your questions, then look back at your questions to determine if you have written any leading questions. If a question is written in a way that presupposes the answer, rewrite the question. It has to be a neutrally written question as much as humanly possible. You do not want what I call the do you still beat your wife problem. You do not want that problem. Because otherwise, you’re going to be given bad data. Data that validates your suspicions is bad data. Data that causes you to rethink your entire business might also be bad data. It’s data that outs the truth. Whatever you think the truth is That’s what it is. And I know we can get into a huge epistemological argument about what is truth and can we truly understand anything in the world? You’ll know it. So that’s rule number one. I always write something and I kill questions if they aren’t working out. I a lot of the time I rework questions so it keeps you from providing neutral answers. There’s a lot of tactical stuff out of this. Even numbered responses. There are no five Like, rate from one to five responses for me. Because what happens? Fucking everyone rates a three. What does a three tell you? You failed as a survey maker. A three tells you you wasted your time and your money and your resources. Or make it a thumbs up, thumbs down.
I was about to say: like, do you hate it? Do you love it? Force people to make that. Yeah. That choice.
Determine what’s required, especially with text fields. People hate writing responses. They hate it. They’ll often just write the word okay or yes or I like it or no in free text fields that are clearly meant to be a multi-paragraph essay. So it’s the honest is on you to reduce the cognitive load as much as humanly possible. One thing I love u using is Write as much as possible. There is no limit to this field. Oh, God, yes. I love writing that. And I do that during times when it’s clear this is the kind of response that would be best served by your therapist. I love doing that, especially on anything about running an independent business. And it’s like, tell me about the struggles you face in running your independent business. Brother, we have a podcast for a reason.
You know, our therapy session. We’ve got 20 episodes in the queue here to record. There’s a rich amount of data you could collect there. Right.
So, you know, in situations where you’re going to get the fire hose, encourage the fire hose. You want to be acting on too much. It’s far better than the alternative. Those are a lot of the things that I do that I mean, those are very open-ended. I try and and Check my writing as much as humanly possible. And I think that’s the biggest one: is you know, look at look at the way Pew writes their surveys. They randomize their questions. They randomize like the order of phraseology in their questions to make it as neutral as humanly possible. go through their actual surveys and see how they do it. Because or like political surveys, like political polls. We’re dealing with an election in the United States right now. Look at the way they phrase their questions. You will learn a lot about how to phrase good questions. It is very hard to ask a good question. It’s so much harder than you think it is.
That’s the end of my statement. And I guess the last question to wrap up on is. You start research, you collect data, you talk to people, you end up with a strategy, you use a methodology, you’re iterating on this research. How do you act on it? How do you put it into practice?
This is where the magic is. I don’t know. The better answer, if I were a designer, is it depends. Designers love saying it depends. But it it really like I don’t know. I look at the data, I look at everything, I say they’re getting tripped up here, let’s fix it Right? You can look at data and determine it. And I’m sure that what I’m doing is fraught with cognitive biases and all sorts of issues. Read up on cognitive errors so you make sure you’re not done and Krugering yourself. And just go. Have a conversation with other people. The best thing you can do when you’re synthesizing research, there’s one thing I’ll say: don’t do it alone. Make it a conversation with other people because everybody’s going to have an opinion. They’re all going to look at, it’s a Rorschach block. They’re all going to look at the data and see different things. This is one situation where a rare moment, I really wish I weren’t solo. Because then I would have more people to talk to about research. So what I end up doing in that situation is running it past like other colleagues, stuff like that, trying to anonymize it as much as humanly possible. So, I don’t have to deal with privacy issues. Or I’ll read a lot of books that’ll help me out. But acting on research data is a fine art, and I really don’t have a solid concrete answer. There’s no one weird tip on it. It depends on the data you get in. It depends on the context. It depends on the project’s needs. And it’s a unique snowflake every single time. I have no good answer for you.
In my case, with outreach, with public relations campaigns or marketing strategy, I found like the acting on the research data is Taking the data and saying, like, there’s a lot of information here. There’s more information than we can necessarily act on, given our strategic priorities: increasing traffic, building relationships, something else. What can we tease out of the data that gives us insights to act on? Okay, let’s move forward. It’s going to be a test. We don’t know whether it’s going to succeed or not, but let’s inform our decision-making and our Creative process based on this research data. And let’s track results, figure out what the KPI, the key performance indicator is, figure out what the metric that makes the most sense is, and a month later say, are we winning or are we losing? And based on that, like what was a success, even if we’re losing, and what do we want to change? And let’s go back to the research data and You know, tease out something else and iteratively work through it. Just go, in my mind, at least, and slap me if I’m wrong on this, going through a research process doesn’t give you the absolute answers you need for a campaign or a project. It just gives you data to draw on and act on as you move forward. You could take the research data, implement a test in any discipline, and maybe you win, maybe you lose on that test. It doesn’t mean the research data was wrong. It just means that the conclusion you drew from some part of that data to inform the test or the campaign, well, it didn’t perform the way you expected. Okay, let’s iterate on that.
Your response was much better than mine. I feel like it’s just like for me, it’s like a matter of looking in the mirror. It’s like, oh, my hair is messed up today. Let’s get a comb and fix it. Oh, I have a giant Zid on my nose. Isn’t that great? I’m 34 years old. Like, that’s just, I’m looking at it and thinking, where are the problems? How do we fix them? And you’re a little bit more, I think, measured about it. And either is fine, really. It’s something you get better at over time, and especially if you’re like in-house at a business. Or you’ve had like one client for a year or something like that. Thoughts so much better. Um ‘cause you’ve learned the The contours of the data. You know what kind of data you’re going to get in. Okay, well, what do you do with it? How does it change over time? Can you compare it with something that happened six months ago? Because it’s probably different.
All that’s super valuable. Yeah, and I’m blessed that with my current clients, three of them I’ve had for close to a year now. And so It hits that tipping point where I understand their business on a much deeper level. And research is still very important, but I’ve internalized a lot of the information about the business to the point where I’m able to research in different and new and interesting directions. It’s no longer. Complete unknowns for me. I’m starting to understand where the limits of my knowledge are and use research to better fill in those gaps. So when a new project comes down to the line, it’s not like Start from square zero. It’s like, okay, I have a good understanding of the messaging and the concept and the strategy for the business. This is a tactical execution on a project. What don’t I know about this? How do I research it? What holes do I need to fill in? And how do I iterate forward on that front?
Yeah, yeah, exactly. Exactly. It’s you’re never done. There’s no end point to research. It’s like saying there’s an end point to like listening to your friends or your partner. Like, I’d hope not, God. And you don’t have friends or a partner anymore, and that would suck, you know? Like, so if you’re, you know, if you’re starting to get Fatigued on it, figure out a way around that. Figure out a way to attack it from a different angle. And you’re never done. Design is never done. You’ll launch this giant redesign, and it’s like, okay, now get back to work.
Mm-hmm. Mm-hmm. Entirely, entirely. And and Yeah, connecting to that get back to work idea. Is there or should there be an end point to the research? Or is it possible to get stuck in a research loop or a research hole and feel like, I got to keep researching, I don’t know enough to act forward on this now? And there comes a point where you have to say, like, I might not know everything, but I know enough to get started. Let’s see what we discover with this research.
Yeah, yeah. I mean, there are limits to what you can discover with research, but then do different research. Don’t do more of the same research because you’re probably not going to get a whole lot out of it. Just do different research. It’s fine.
One tip/slash trick I wanted to mention on the episode, connecting to the idea of research, a tactical thing that I love sharing. For any audience members who are doing something like copywriting or trying to understand the voice of their customer, one thing you can and should do for research is go to Amazon and look at Some of the top-reviewed books in that subject. So let’s say you’re, I don’t know, in the leadership space and you want to understand the language or the problems that People who buy leadership books are facing for content creation, for messaging, for whatever reason, look at the reviews. See what exact phrases people use to either praise the book, this book helped me understand how to lead my team and solve my accountability issues, or To diss the book. This book was terrible and did not help me solve this problem. Read all the two-star reviews. Yeah, read all the two-star reviews, read the one-star reviews, read the five-star review Please throw out the 3-star and 4-star reviews. But those two-star reviews, they’re going to tell you the problems people are looking to have solved for them and the exact language they use to describe it. You could use that to inform your content creation process. People keep talking about like, none of these books taught me how to X, my, Y, do, and the Z. An article about that and use the exact language that people use in those reviews in your headlines, in your subheadlines, in your copy, in your calls to action, in your articles, in your subject lines. Because that’s how your users, that’s how your customers, that’s how your people, your tribe, your rat people actually talk to each other. Appropriate and reuse that language, and it will be one of the most valuable research activities you can do.
Absolutely. Absolutely. I think it’s a good place to leave it there. You know, one thing that we should try and avoid is being boring. You should rate our podcast either five stars if you got anything out of this episode or one star if you didn’t. Like, don’t three-star a podcast. That would make me very up. No, the last thing I want to do is be boring. So you should give us at least five or one star. That’s it. There are no other options. Thumbs up or thumbs down.
Go. And the best one-star review, we’ll probably put on the website.
Yeah, we put a good review on the website recently. I need to put a bad review. I need to put like a these people weren’t for us review. Like they wasted my time. I hope we didn’t waste your time. They did not teach me how to make enough money online. I only made $18,000 with their advice. I wanted more. How dare they
Notes
- Why is research important?
- How does research fit into business?
- Specifically, how does research fit into each of A/B testing, IX design, Outreach, Marketing, and SEO?
- What methodology do we use for research?
- How do you iterate research?
- How do you throw away research?
- How do you make a good customer survey?
- How do you act on research data?