For the last 6 months, the ID90 Travel team has been focused on building a new app. The primary driver for this product was the shift in traffic and noticeable downfalls of our current app. After deliberating what technologies to use (Ionic, Cordova & Angular), we began building. After 3 months, we decided we had an MVP we could elicit feedback from real life customers. So we decided to run a series of listening labs.

A listening lab, as outlined in Customers Included, is a qualitative, open-ended, one-on-one customer observation session. Unlike focus groups, listening labs are not prone to confirmation bias nor direct customers to follow a set of defined steps. Listening labs attempt to capture the customer in their natural state while using your product. A similar concept is outlined in Sprint, a book on testing products with customers before writing a single line of code. As you one interviewer runs the listening lab, you’ll have your entire leadership & engineering team watch the live session and take notes of their findings. Below you’ll find recommended tools you can use to successful record and broadcast your interviews.

Why host a listening lab?

Listening labs allow us to have one-on-one conversations with customers in an open format. We encourage the customer to be honest and vocal. The team asks a series of simple, open-ended questions that do not trigger binary yes or no answers. The combination of this allows us, the engineering and product team, to understand flaws in our product’s design.

1. Schedule 5 Customer Interviews

Before you can run a successful listening lab, you will need to schedule five customers on the same day for 1-hour sessions. An interview usually lasts about 30-50 minutes, so 1-hour sessions are perfect.

You may be wondering why only 5 customers. Professionals advise five customers, as researchers have found there are diminishing returns after the fifth interview. You’ll notice it too once you complete all the interviews.

To find customers, you may either contact existing customers or reach out to potential customers. In the latter scenario, you’ll want to use a tool like Craigslist and screen customers with a survey. You’ll want to incentivize customers to interview in either scenario. I usually advise a $50 or $100 Amazon gift card.

2. Setup Up The Listening Labs

As mentioned above, you will need to broadcast and record the interview. Your engineering, product and leadership team should be in another room watching the interview on a large TV. You will be in the room with the customer with an arsenal of easy to use tools. The tools you will need include:

  1. Microphone
  2. Laptop
  3. Google Hangouts
  4. Reflector (iOS Apps)
  5. Vysor, QuickTime, Photo Booth (Android Apps)

Although laptops typically come with microphones installed, we advise investing in a high-end external microphone. This should cost less than $100.

Google Hangouts: you’re going to use this tool to video conference the interview so your team members in the other room may watch. You will be sharing your screen and your team members will all be on mute.

Reflector (iOS Apps): this tool allows you to mirror what the customer is doing on the phone onto the laptop screen. It will also allow you to record video via your laptop. The goal here is to not only record the app interactions, but also the customer’s reactions in real-time. You can use the free trial of this tool or purchase it (it’s super cheap).

Vysor, QuickTime, Photo Booth (Android Apps): unlike Reflector, Vysor is an android app that only allows you to mirror the phone onto your laptop screen. As such, you’ll need to use QuickTime to record the screen and photo booth to record the customer’s reactions.

3. Get to Know The Customer

So you have 5 customer interviews scheduled. You have all of the equipment and tools necessary to run successful listen labs. Now all you have to do is get to know each customer before starting the actual interview.

When I say getting to know your customer, I mean making them comfortable with some small talk and asking questions about themselves. The key here is to make the conversation about the customer. First, greet the customer and thank them for their time.

“Hi John, thanks for coming in today. We really appreciate you taking the time to test our new product.”

Then, you’re going to want to ask them to answer a few open ended questions. For instance, let’s say you have an app that helps educate and sell artisanal coffee to customers. You could start off the interview with the following questions.

Common Preliminary Questions

“Before we begin, could you tell me more about yourself and what you do for a living?”

After that, you can ask questions that lead up to your app:

“And how long have you been doing that?”

“What do you do when you’re not working?”

“What kind of coffee do you usually drink?”

“How often do you drink coffee?”

“Have you used any apps or websites to help you learn and find new coffee brands? Which ones?”

“How often do you use these apps or websites?”

“What do you expect these apps to do? What do you like or dislike about them? Do you pay for them? Why or why not?”

Asking these questions will do two things. First, it will allow the customer to open up and become comfortable. This is important to ensure they can be vocal and frank with your product. Secondly, it will allow you to paint a picture of what persona this customer falls into. This will allow you to develop a sense of empathy for the customer and relate them to others who may be in the same persona group.

4. Run The Listening Lab

Once you’re done asking the preliminary open-ended questions, ask the customer if they are ready to test out the app. You will want to inform the customer that this is a beta version of the app still under development, so there may be instances where everything doesn’t work 100%. Inform them you’ll let them know if something doesn’t work and how to proceed.

In the coffee example outlined above, ask them to open up the app and shop around as they would normally. Remind them to think aloud and to tell you what they are thinking. Give them open-ended tasks that are not necessarily direct in nature. Also, be sure to ask questions about the experience or design such as:

“What is this? What is it for?”

“What do you think of that?”

“What do you expect that will do?”

“What are you looking for?”

“What would you do next?”

This portion of the interview should take approximately 30-minutes. Once you’re done, do a debrief and ask them a few final questions about their overall experience with the product. Questions like:

“How does this product compare to what you do now?”

“What do you like about the product? What do you dislike?”

“If you had 3 things you’d want like improved for this product, what would they be?”

5. Find Commonalities Across Notes

You’ve finished your five listening labs. Your team has feverishly jotted down notes about the customer, the moments they loved and the moments they were confounded. Now it’s time to synthesize notes and find trends across customers.

You’re going to want to see where most of your customers got stuck or confused. You’ll also want to focus on how they went about completing their tasks. Take notice to how long it took for customers to find what they were looking for and pay attention to what they said when asked probing questions.

A good way to go about this is to categorize your screens and functionality. Did customers readily locate functional elements such as filters? Were any forms overwhelming? Did the flow of your process make intuitive sense? Where customers able to find their history, important profile information and like items?

Make note of not only the number of times a customer had an issue for a given screen & function, but also prioritize in accordance to the business importance. Filtering your results is important, but not as important as a confusing checkout form.

6. Rinse & Repeat

Your team will now have sufficient evidence to support redesigning particular screens and/or functional designs. Take this time to meet as a team and have white boarding sessions. Identify the pitfalls and start throwing ideas on the whiteboard to see feasibility. Also, ensure to research how other websites or apps handle the problem. You may also find various studies published by institutions that focus on Usability like the Baymard Institute. The solution may be easier than you think.

After you’ve implemented a new experience to help with the previously recorded pitfalls, go through the process once more to ensure you’re on the right track. You may find that while the new design resolves a previously reported issue, it may give rise to new unforeseen roadblocks. This is why listening labs are so important: as your app evolves, you’ll want to ensure you’re still appeasing your customers with an experience that delights them.

Conclusion

I recommend running listening labs like these with customers at least once a quarter. It would be preferable to run this exercise once a month or after every other sprint cycle. This practice is an easy way to get valuable qualitative feedback from customers so you can gear your product in the right direction.