sd

Product Management

How to Run a User Test?


There are times when product owners with brilliant product ideas jump directly into turning it to reality. They invest time, effort, and often a considerable amount to build the product. They believe it’s going to be a hit. And far often than not, these products fail miserably.

Why?

Most probably because they ignored the user validation process, the product owners thought the idea was brilliant, but end-users begged to differ. A user test is all it takes to validate your ideas and build a product that will be a product-market fit.

What is a user test?

User Test is a process where we test the solution to check whether it works for the user or not. By performing the user test, we are making sure there is no friction for end-users to adopt the solution. We are creating a path for the user to use our product again.

For instance, while testing for user onboarding, we are trying to validate the onboarding process. It is just about onboarding. Hence, we experiment with only a single variable. If other variables are not isolated, it will be difficult to understand the cause and effect. Ultimately, causing difficulty in tracking the success metrics.

The user test can be carried out in different stages of product development. From idea generation to growth feature, it is carried out to validate assumptions and hypotheses about a product. It helps align your product requirement with users, which in the long term will maximize the benefits of the company.

Who do you want to test the product with?

Remember that you want to test the product with your target users and not everyone. Your friends and family are not your target audience. Select up to 5-7 people to test the product. The sample size might seem small, but it is enough.

As per the Goldilocks principles, you will receive 85% accuracy no matter the size. And for us, 85% of validation is good enough to move on to the next phase of development.

Make sure to select a sample from diverse groups. Otherwise, your result will not be scalable. Instead of creating a product for your target audience, you will end up with custom software for a niche market.

How to run a user test?

  • Start with a problem and a hypothesis as to why it is happening
  • Define the success criteria such as 4 out of 5 people should validate the assumptions, increased activation rate, etc.
    Then, take 5-6 people who are also your target audience — set meetings- virtual or in person.
  • During the meeting, start with a conversation. Do not make the users feel like they are in a test. That will surely make them conscious. Plan the test in a way that feels natural to them.
  • Let the users use the product and observe how they interact with the system. Observe their mouse control, encourage them to speak their minds out. If it is a remote interview, ask to share screens or recordings of product use.
  • Don’t guide the user. Let them find their way to the features.
  • All the while, observe and make notes on their body language, facial expression. Make them feel comfortable.

Validation after the user test

The validation process after the user test can be divided into:

1. Qualitative

In qualitative, if XYZ is a problem, we assume as to why it is happening. Create a hypothesis around it and also define success criteria like 4 of 5 people believe our hypothesis.
If the success criteria are met, we consider the problem is validated.

2. Quantitative

In quantitative, we use metrics to compare and analyze the before and after numbers. We decide on a metric before the test and later compare the two results to check whether the hypothesis was correct.

Leveraging Metrics for user test

We can also use the AARRR metrics to prioritize features to test.

For instance, suppose the acquisition rate is high, but the activation is low. You can analyze the FullStory to see what is not working. The value proposition might not be clear. Or there are too many fields in the signup form. By selecting one of the hypotheses run user tests to see if your assumptions are correct.

Once we have tested it with 5-7 people, we can also run an A/B test to test with a cohort and measure it. The features then can be broken down into epic if it is vast, added to the backlog, and prioritized. We have to validate every element before going to the build phase.

Things to remember during user test:

  • During a user test, think of how a product can fail rather than succeed. Having this mindset will allow you to view the outcome from a different perspective. It will give you ideas for improvement.
  • The people will take actions in three cases:
    1. The frequency of the problem
    2. The severity of the problem
    3. The urgency of the problem
  • Does your target users feel the same about the issue you are trying to solve?
  • Ask practical questions rather than hypothetical ones that relate to the product.
  • Try to analyze from the user’s behavior what can be added to the product that can maximize the benefit. Also, essential to understanding what is not in the product that can benefit the users.
  • The environment where we carry out the test can also alter results. Try to carry out the user test with the least interruptions and diversions.
  • Always remember it’s about the user rather than the feature, product, or company. They are the ones whose problem you are trying to resolve.

 


To unpack more questions related to product development dive into our Product Podcast.

Footer CTA Product Podcast


More in Insights

Why Businesses Must Focus On Unit Economics? Product Management

Why Businesses Must Focus On Unit Economics?

Many businesses can predict whether they are going to make a profit or not. To do so we use a

Read more
A Deep Dive into Product Management Tips and Tricks for the Consulting World with Lewis C. Lin Product Management

A Deep Dive into Product Management Tips and Tricks for the Consulting World with Lewis C. Lin

Be The Greatest Product Manager Ever: A Deep Dive into Product Management Tips and Tricks for the Consulting World with

Read more
How to Implement the CIRCLES Framework into a Services Company? Product Management

How to Implement the CIRCLES Framework into a Services Company?

As product owners, we think of brilliant ideas that can create an impact. We strategically build a team and start

Read more