Useful questions for user tests

August 11, 2020
📓 Article

We’ve already covered useful questions for customer interviews. But how can you get insight on your content, and your site, from people who you don’t know yet? We really like services like for this, and in this article we’ll suggest some questions you can ask during a user test to get useful feedback on what you’re doing.

By the way, this article covers a specific type of user test – a remote, moderated test. This type of test is conducted by the subject, using a list of questions you provide. Because you won’t be there to ask questions when conducting this type of test, it’s critical that the subject be encouraged to talk through why they are behaving in certain ways, and how they are thinking about their interactions with you.


Be sure to screen out testers who aren’t in, or related to, your target market. If you provide an accounting solution for small businesses, for example – you might want to select for people with some familiarity with competing products, or who have job functions that are related to accounting.

The 5 second test

In the 5 second test, you want to show users your site – most likely, your homepage – for just 5 seconds and see if they can understand what you’re about. This is a great test to see if you have a clear value proposition that people who don’t know who you are, can understand. Once users have had 5 seconds to look, specific questions you can ask include:

  • What do you remember? This is helpful for understanding if your value proposition is clear – and also for understanding if there are distracting (or particularly memorable) elements on your site.
  • Who’s this site for? A good test for seeing if the target persona is easily detectable. This is a question your visitors will typically ask themselves when they land on your site – and if the site isn’t for them, they will leave more quickly.

Home page exploration

Ask your user to take 30 seconds to get acquainted with the homepage. This attempts to simulate the experience a user will have if they become seriously interested in your product. Good questions to ask here include:

  • Without leaving the homepage, what are your initial impressions of the website? Explain your answer. This is a good way to assess a visitor’s initial level of trust in your website, when they arrive. Asking a visitor the reason for their impression is helpful, too – since the answer may point to specific issues you can fix.
  • Please describe what our product does. This helps you figure out whether your value proposition is easy to find, and if so, whether it’s easy to understand.

Screen Shot 2020-08-16 at 8.55.50 PM.png

And you don’t have to be fancy to be clear.

Screen Shot 2020-08-16 at 8.55.31 PM.png

Task accomplishment

The buyer journey on a B2B website is extremely complex. Your visitor is probably on your site to research your solution, and may also have the goal of being able to bring information back to the rest of their team. So, ask your users to accomplish information-gathering tasks that you commonly expect users to accomplish on your site. For example:

  • Determine whether our product supports Linux.
  • Get information about the ROI of our product.
  • Find information about how to most efficiently deploy a new bid management system.
  • Tell us how you would contact our sales team.
  • Explain how we compare with Competitor X.

Be sure to have users talk you through what they are doing, and why they’re doing it. “I’m going to click on ‘Resource Library’ because I think that’s where you keep whitepapers on this topic.”

Test your competitors’ sites

Lastly, consider running user tests against your competitors’ sites, too, to see where they’ve done a better job of helping visitors get the information they need. Whoever provides information is likely to be able to frame the conversation.


User testing is a relatively inexpensive way of seeing problems on your site that you don’t notice. And they can be a great way of convincing others on your team to take action – once you’ve conducted 5 - 7 tests, send out a highlight reel so everyone can see how real users interact with the content you’ve created.

Principles for driving content performance

July 24, 2020
📓 Article

How do we see forward-thinking marketers drive content? There are four principles that we’ve seen our clients use to be successful with content performance and SEO.

  • An organic search strategy that’s well-executed today can improve your marketing across all channels, forever
  • Marketers should build an audience that they own – not search engines
  • The best content libraries are treated as a product that’s continually improved
  • Underlying all successful marketing is an efficient idea-to-content pipeline that makes generating high-quality content part of the day to day operation of your business

1) Build an SEO strategy that improves your marketing everywhere

To be successful in organic search, you need a strategy. “Keyword strategy” is often made to sound complicated, but fundamentally it’s pretty simple – find topics (not just keywords, topics) that are relevant to your business, with a worthwhile balance of competition and volume. Here’s how we do keyword strategy, by the way, plus our template.

Once you have a strategy, you can pursue it using tactics that improve your marketing everywhere, not just on search engines.

  • The most important of these is to generate high-quality content – articles that have enduring interest for your customers, that respond to their questions and help them get more done, and that are additive to what’s already available to them.
  • This content has a good chance of performing well in search, but it’s also critical to distribute this content far and wide – we call this a “link distribution strategy”. One component of this might be backlinks, but what really matters is a process to get this content in front of as many people as possible who it can help. That means making sure it shows up not just on social, but also on sites that are relevant to your community, on Q&A forums, in relevant articles that other people have written, in emails coming from your sales reps, and more.
  • Since this content is relatively hard to create, you’ll get the most out of it by reusing it as much as possible. A blog post becomes a webinar, and it’s a chapter in a whitepaper, and so on.

Following these practices, you get your organic search strategy to reinforce all the other places that you’re marketing. Even fixing technical issues on your site can be helpful for everything else you do if you focus on things like improving conversion rate or fixing user experience issues.

2) Focus on building an audience you own – not Google.

A huge percentage of traffic is driven to sites via search – between paid and organic clicks, some estimate more than 70%. This isn’t necessarily a problem, particularly not in the short term. On the organic side, that means changes in algorithms influence whether people see your content. Sometimes that can be really good, but sometimes it isn’t. On paid channels, if you stop paying, you lose access.

(This isn’t just a search problem, by the way. It’s a problem with any channel where you rent access to your audience, and that could include content syndication, paid email lists, social, and many other channels.)

If, on the other hand, you spend time building direct connections with your audience – via newsletter subscriptions, targeted email, freemium trials, LinkedIn connections, and so on – you can talk to those customers anytime you want, without having to get past Google’s or anybody else’s filters.

Aside from revenue, growing a direct audience is the most valuable thing a marketer can do. It’s not only a source of future customers, it’s also a source of message propagation, because the people for whom your message is useful send it out again to other people who might want to join your audience.

3) Treat content as a product, with a library that’s continually maintained and updated and acts as a resource for customers.

As compared to one-off blog posts, a product is a collection of parts (individual content pieces) that work together to help your customer level up in their job. Here’s an archive of the Drift blog from when they were just a couple years old, to show you how this can work. You can see how all the posts work together to help their prospects and customers to do their jobs better. Posts include:

  • How to define the relationship between product managers and marketers, with a link to a Slack community
  • What Relationship Marketing is
  • Growth marketing framework
  • Explanation of how UX applies to marketing

Your content is a library for your prospects and customers, and the people they work closely with. For all of those people, what is everything that they need to know to do their jobs well, preferably with a focus on the use cases or tactics that your product enables?

4) Build efficient, effective idea-to-content pipelines that make generating content part of the day-to-day, not a chore

Marketers often focus on how to come up with content ideas, but you also need a good way to convert your ideas into published content pieces. We call this the idea-to-content pipeline. The key to the idea-to-content pipeline is that, as much as possible, you want to avoid writing things from scratch. Ways to get around this include:

  • Picking topics that you know you can write about. Looking at the keyword strategy advice we give above, an additional consideration might be the amount of effort involved in creating something that addresses a particular topic.
  • Creating an outline first, and populating it using data. For example, when we write an outline on a particular topic, let’s say “carburetor repair”, we’ll use Google’s search suggestions, or other topics we want to write about, or what’s popular on social for a specific hashtag (#carburetors) as suggestions for sections to write. As a bonus, content tends to perform better on search if it hits related topics that Google considers important.
  • Interviewing your customers (and colleagues) regularly, with questions that help populate missing pieces in articles. As a bonus, some of these interviews probably could be, and should be, webinars or office hours.
  • Repurposing as much as you can. If you think about your content library as a product, can you take other pieces of the product and combine them into something new? For example, if you are writing an article on “cutlery management” for your restaurant clients, can you insert existing content about “bamboo vs. plastic”?
  • A notebook is really helpful. We keep pretty good track of all the conversations we have, and the same themes tend to emerge as we look backwards to conversations we’ve had weeks or months ago. These themes are attached to specific wording and notes that we reuse in our blog posts.
  • Having a good process for getting content up on your site is really important and often overlooked – most people think that the CMS that powers your site is a technical decision, but in large part, the CMS you choose should be governed by the speed at which it lets you get content live.


Designing great posts based on keyword research is only the beginning of content performance. Think of your content as a product, and use it to build direct relationships with your target audience. Over time, individual posts will add up to a content library – a resource that keeps customers and prospects coming back.

To build this content and momentum in a sustainable way, break big campaigns down into daily tasks. With an idea-to-content pipeline, this can all be one fluid, creative process.

Good questions for customer interviews

July 16, 2020
📓 Article

To create content that performs, it’s critical for writers to understand how customers perceive your product, what problems they use it to solve, and what problems, more generally, they have.

(In shorter words: writers need empathy.)

A great way to do this is by talking to customers. In the vast majority of companies we’ve worked with, content producers, and often even marketers more generally, don’t do this. Common objections include:

  • Not having time
  • Not wanting to bother customers
  • Not knowing what to ask

The first two are easy to deal with. The time you take in building empathy with customers is paid off by the time you’ll save creating the right content, and really understanding where your customers are coming from. And as for not bothering customers – they would love to talk to you about your product, as long as they feel like you’re listening. Relationships with your customers, like trust, are a positive-sum game.

So what if you don’t know what to ask? Here are a few good questions to start with.

Tell me about yourself

This is a quick one, but you want to make a connection with the person you’re interviewing. What’s their role, how long have they been in it? How do they describe the company they work at? Useful context for understanding where they’re coming from.

Before you start the interview, you should also take a minute to tell them who you are and your role at your company. And some general tips: show your engagement with the interviewee, and take it slow since often the most interesting information comes after a moment of reflection.

What do you like about your work? What do you dislike?

Start listening here for things they want to do more of – sources of value for them in their work – and pain points. If you’re interviewing an SDR who uses your product and they tell you that the most unpleasant part of their day is filling out reports, you know that the ability of your product to automate reporting is useful to them.

Tell us how you heard about our product

Did they hear about it from a friend, see it on social, or read some interesting article that led them to it? In any case, this is critical data for your content distribution strategy, and you’ll want to share it with your demand gen team, too. (Or maybe your customer can’t remember how they heard about it; that’s useful information, too.)

For extra points with your marketing ops team, see if your customer’s memory of where they heard about it is the same as what’s reflected in your lead gen analytics.

What are you using it to do?

This is the first question that will really help you home in on what you should be writing about. Not only does this answer tell you what problems your software solves, but more importantly, it tells you how your customer thinks about the problem your software solves.

You might get a really simple answer: “We use your software to manage our inventory.” But that gives you a chance to dig a little bit deeper. “Tell me more about that process”, “How did you do that before you started using our software?” or similar questions will reveal details about the customer’s goals and experiences that you can use to write more useful content for them.

What’s your favorite part of our product?

Share this answer with your product team. But also, the language here is useful for your landing pages and calls to action.

“What is your favorite part of using our CRM?” “Well, I love how flexible it is. You can basically do anything with this software.”

This customer (and others) say that they love the flexibility of your software. This leads to:

  • A series of how-to articles on all the interesting things you can do
  • A survey of top things that your CRM is used to do
  • The use of the “flexibility” theme in your ads, product page copy, and landing pages

If you had a magic wand, what would you change?

Share this answer with your product team, too. From a content perspective, the opportunity here is to know what to avoid. For example, is your search painful to use? Maybe take “powerful search” out of your landing page bullets.

If you were to describe our product to a colleague, what would you say?

This is a key question for your organic search strategy. How do your users talk about your app? What do they think it is? You can target this phrase in search. But also, if you’re hearing answers that are not what your product does, that gives you an opportunity to write content that helps users better understand what they should be using it for.

“I usually describe it as a tool that makes it easier to schedule appointments.”

Of course that means you should make sure you’re targeting how to use your product to make appointment-setting easier. But it also means that you can write some other posts about how your product maximizes customer satisfaction, or facilitates customer engagement via instant videoconferencing.

Any other comments?

Letting your customer talk will give you some useful insights. They may also have a problem that you can easily solve for them – a misunderstanding about how a feature works, for example, or lack of awareness about something that’s available to them.


Investing in customer interviews will generate a huge payback for your content marketing efforts. Even one interview every week or two will help you build empathy for your customer. Becoming a skilled user interviewer takes time, and there are lots of places to learn more; Nielsen Norman Group is a go-to for us.

Building trust with your site visitors

July 15, 2020
📓 Article

Marketers spend a lot of time figuring out how to get attention. We pay for it on Google and LinkedIn and Instagram, we think about whether our search results will end up at the top of someone’s screen when they are trying to solve a problem, and we send lots and lots of email with cute titles. We use lots of fun gimmicks – sometimes ones that have nothing to do with the brand or what you offer – just for the attention and the chance to continue the conversation.

It’s competitive, and the competition is exhausting. Attention is zero-sum. There’s a fixed amount, and to get attention for yourself, you need to take it from someone else.

Sometimes looking for attention works, and it’s certainly an important thing to know how to do.

But there’s a big part of the story that marketers often miss – building trust. Trust is needed for any transaction to take place, and the world’s biggest brands are often those that are the most trusted. Trust is:

  • Positive-sum. When you build trust with a customer, it’s easier to build more – versus attention, which you get a limited amount of. And when I build trust with a customer, it doesn’t mean there’s less trust for you. In fact, we can both benefit at the same time.

  • A way of reducing friction. When you build trust, the right customers go from wondering, “how can I avoid buying from this person?” to “I bet this person can solve my problem, and I want to work with them to make that happen.”

  • Key to customer satisfaction, especially for complex products like software. The path to getting value out of them is long and circuitous, and it only begins with the sale.

So how can we focus on building trust?

Make promises, then keep them

The major way to build trust is to make promises, and then keep them. This manifests itself in a lot of ways, including consistency, value, and authenticity. Some examples:

Explain clearly what your product is about on the front page. When companies make really vague pronouncements on the front page, it’s a missed opportunity to make a promise about what you deliver, and how. And it often confuses visitors or turns them off.

Here’s an interesting example. Is the heading clear? Should their subhead be the heading? If you had never heard of this company, would this make you want to buy?


Make it easy to access useful information. This can mean a lot of different things, from having a clear and consistent navigation bar, to having clear topics in your content library that are navigable and correspond with your visitors’ problems.

The content library from Nielsen Norman Group, a UX consultancy.


Be consistent with your brand. This can mean simple stuff like making sure your design is up to date and helps your user navigate, rather than getting in their way. More generally, it means making sure that all your communications and all of your brand personality works together (though that’s a topic for another post.)

Price honestly and fairly. Is it easy to cancel? If a user isn’t getting value from your service, can you charge them less or automatically switch them to a lower tier?

We love this “maintenance plan” for a service we recently canceled (left). It’s not available until you try to cancel – which is super-easy, by the way – but it gives the user an option other than “we’re going to delete all your data.” By way of comparison, does knowing how hard it is to cancel the New York Times (right) make you want to sign up?

Screen Shot 2020-07-16 at 5.28.27 PM.png

Make outbound touches useful and relevant to your prospect. Personalization works in outreach. Why? Because it creates trust that someone’s reaching out to you for a reason, and has done their research.

Is this useful personalization? How could it be improved?



What would happen if you viewed your goal as creating trust rather than getting attention? Slower growth maybe, at first. But ultimately – much more durable, valuable relationships with your customers and prospects.

A/B testing for startups

July 14, 2020
📓 Article

Generally, we tell clients that they need 1,000 conversions for a reliable A/B test, for each variation they’re going to test. (There are actual calculators you can use, too, but this is a rule of thumb.)

That means if you have, for example:

  • A landing page with 50,000 visitors every month, and a 5% conversion rate (= 2,500 conversions)
  • Or an ad with 250,000 impressions per month, and a 1% click-through rate (= 2,500 conversions)

You’re going to be able to run 1 reliable test each month. Of course, there’s a lot more to the story, and we recommend using a calculator like this one to know for sure. One other big factor is the size of the uplift; if you have larger differences between the test group (say they have a 10% conversion rate) and the control group (say they have a 1% conversion rate), the difference is also easier to detect. There are other parameters you can play with as well.

But the overall point is that it takes a lot of traffic. As you grow larger, you can run more tests! A million views of your homepage every month, with a 5% conversion rate, means 50 tests a month – and you can really get into things like buttons, form fields, copy, and more.

But what if you don’t have that much traffic? Is an A/B test still worthwhile, and how can you make it count?

Maximizing the usefulness of A/B testing

Given how few reliable A/B tests most marketers can run at a time, we suggest a few important practices to make sure they’re effective.

1) Test big. Testing slightly different landing page copy, or button size, or font color, is interesting! But ultimately, these tests often yield smaller improvements that take a long time to show up. Worse, by the time you’ve completed the test, or shortly thereafter, you’re embarking on a redesign or a new campaign that means you have to throw out your test and start again.

Instead, test an entirely different landing page design across all of your landing pages simultaneously. Try a completely different value prop on your homepage. Hide or show pricing in your nav bar. Hide or show live chat. Try to make big changes, see what happens, and use the results as evidence not just for marginal improvements in performance, but for significant changes in how you talk about, position, or promote your product.

2) Test all the way through. Your ads are a great place to test – super-easy to try different languages, instant learning about what resonates, and usually, something like click-through rate is a faster test than form conversions.

But in addition to testing click-through rates, you probably have a goal of converting your visitor. So you need to test conversions as well to see if your ad copy is simply drawing in lower-intent visitors more efficiently, or if it’s truly doing a better job at positioning you to prospects who would be interested. (You don’t have to A/B test your landing page in addition to your ad, though testing an ad in combination with a landing page might give you a more powerful signal.)

(If you’re an ecommerce business, this is a lot simpler, of course – and effective e-commerce tests do generally track all the way through to revenue. This point is directed mostly at B2B companies with a more complex sales cycle.)

3) Get the fundamentals in place before you test. A/B testing is useful, but talking directly to customers – and perhaps even showing them a landing page and soliciting their feedback – might be worth prioritizing. (And that approach will definitely give you more useful feedback.) There may be other fundamentals you need to work on first, too. How’s your design? Is your page showing up in search? Does it have a clear value proposition?

4) If you are going to A/B test, do it as a program, instead of as a one-off. Bake it into your process to always test your email subject lines, for example, and then choose the winner as the final send. By doing this, you’ll get better at testing, you’ll learn more, and your ultimate results will be a lot better.

5) Don’t hack your own test. Choose a timeline or an endpoint for the test – let’s say 1,000 actions – and then stop the test there. And don’t stop the test until you reach that point. Ending tests prematurely when a desirable outcome has been reached, even if that outcome is mathematically significant, is a major reason why marketers get false results from their A/B testing program.

6) Track your test. We don’t just mean keeping track of the results of the test, though of course that’s important! We also mean – what did you learn from each test? Why did you run it? What did you expect to see (your hypothesis), and what actually happened? This can add another layer of learning, since you don’t just learn from the test, you see how it compared with your thought process before you ran the test.

What kinds of A/B tests are useful?

In general, A/B tests should focus where learning will be most beneficial – and that isn’t necessary where you have the most conversions.

  • For example, if you have a page that lets users sign up for a demo, test 2 different versions of the page, with different value propositions, perhaps a description of what happens during the demo, social proof, and so on.
  • Consider A/B testing different page templates, not just individual blog posts or landing pages.
  • Make A/B testing part of an ongoing program, particularly for marketing emails, email outreach, and, if you have enough conversions, for paid advertising.


A/B testing is a powerful method for improving performance, but if you have less data, there are techniques you can use to really make your A/B tests count. In addition to ensuring a statistically valid test, make your tests bigger – more significant changes, a more thorough view of the entire sales funnel, and more consistent testing as part of the work you do every day.

Programs, not campaigns

July 13, 2020
📓 Article

Many marketers think in terms of campaigns. A campaign is a one-off marketing push around a specific initiative, or idea, or theme, or product release – and it usually has a pretty specific goal to drive a certain number of conversions, or a certain amount of engagement1.

If a campaign is successful, you see an increase in interest in your brand, at least for a while. But often, most of that boost is temporary. That’s particularly true in the case of campaigns that have a significant paid component, where the exposure goes away as soon as you stop spending money.

With a few exceptions (launches), we’re not big fans of campaign-driven marketing. Instead, we urge clients to think in terms of programs, rather than campaigns. Programs are:

  • Ongoing, both in execution and in the results they provide.
  • Iterative, meaning that you’re constantly learning from the results and improving them.
  • Value-focused, meaning that you achieve marketing success by constantly increasing the value you provide to people who come into contact with you.

Why programs work

There are a few reasons programs are more reliable:

1. Data is key to effective marketing

For any program to work well, you need lots of data. Click-through rates, visits, time on site, email open rates, conversions and so on. (Ideally, we also get data about revenue, to make sure that the campaign led to increased sales.)

All of this data, particularly revenue data, can only be built over time. Which makes it really hard to learn from one-off campaigns.

2. Evergreen content rules

If you consider everything that goes into producing an asset — strategy, copy, graphics, production, and so on, it’s a significant investment. Rather than thinking about a whitepaper or a blog post as something you work on once and put away, think about your content as something you can improve over time, so you’re constantly making incremental investments in something that might already be working, rather than brand new investments in things that are untested.

And meanwhile, since most website visits come from organic search, your content can be much more effective if it has time to accumulate backlinks and traffic. One compounding post creates as much traffic as six decaying posts.

3. Iteration is key

In the days of direct mail or print ads, you had to launch a campaign, and you couldn’t change it. People still think this way about digital content. But digital allows (and requires) constant iteration and responsiveness. You can always:

  • Update an instructional post with new features of your product that support what you’re explaining in the post
  • Consolidate posts that aren’t performing well with other posts that are getting traction
  • Make adjustments to a post as you improve your knowledge of what you’re customers are looking for, and what problems you solve for them

4. The scale of modern marketing requires a systems approach

This is a little more philosophical, but: today, your content can reach billions of people. It can reach everyone on the planet! It can (and does) reach all of your target prospects, at the same time.

In order to interact with everyone at the same time, you have to create assets that don’t require lots of intervention from you in order to encourage conversion. You have to create assets that are always present, always reachable, and that stand on their own with your prospect.

If you think about your content library as a product, everything you add to it has the potential to make everything else in the library more valuable. And if it’s valuable, it’s out there, working all the time, but you have to think about your content as a system that requires constant maintenance rather than as a bunch of stuff you write.


In general, think of your marketing as a system for building trust with your prospects and customers. That means one-off campaigns are not what you rely on – high-quality information, consistently produced and delivered – is. And you should think about that content as a system that works together, where everything that gets added improves the value of the content library as a whole.

  1. (We at Ercule are not super-enthusiastic about using military metaphors in marketing, of which “campaign” is one, but we’ll come back to that in another post.)