What I learned talking to thousands of early members
When I first joined the Levels founding team back in late 2019, our app wasn’t technical at all — instead, we had humans on the end of every member inquiry.
While we worked on building out a technical product, we outfitted members with continuous glucose monitors and gave them a phone number to text me directly to share their data and ask questions. We called it MikeBot. (Before I joined, DavidBot [our co-founder and head of product] handled most of the conversations.) Each member also had three one-on-one conversations with me: an onboarding call, a midpoint check-in, and a debrief.
If it sounds like it took a lot of time, it did. I would regularly have 40–50 calls in a week. And yet, this process of talking to thousands of early members was worth every second. It’s helped us build a product that better serves the needs of our members faster. Ultimately, every single aspect of our app and experience came directly from this feedback. Even our co-founder Sam Corcos — the most efficient person I know — would take time to join me on some of these calls because the value to the company was clear.
Since then, a lot has changed as far as how technical our app is, but our interest in staying in close contact with members to learn about their experience has not. Because of its impact on growing our company in the right direction, I continued to call every member long after we had an app. To this day, I devote some of my time to talking to members, especially when we’re launching features or running experiments. All in all, I’ve had one-on-one conversations with more than 1,500 members.
Every customer you talk to is an opportunity to learn and improve your product or experience for the next thousand, tens of thousands, even hundreds of thousands of others down the line. I recognize that synchronous time like this is expensive. But, if it’s used correctly, the rewards can outweigh the cost.
Here are some of the things I learned about getting the most out of these customer interviews so you can use them to make a better product.
Let Your Customer Tell You Who They Are
When you’re receiving feedback from a customer, context matters — and it’s something I don’t think enough people consider when conducting customer interviews. Understanding who is giving you any given piece of feedback can help you dig deeper into the why behind their suggestions (more on that in a minute). It can also help you understand who your product is working for (and who it’s not), allowing you to hone in more quickly on your product-market fit and accurate user personas.
Sure, you can — and should — get basic demographic data, most of which you can collect before the interview or may already have as part of your product onboarding. That information can give you some context and ensure you’re getting feedback from a diverse mix of people. We always aim for an even gender split, a healthy range of ages, and a combination of promoters and detractors based on our net promoter score (NPS).
But I like to dig deeper than basic demographics. One of my favorite questions to ask as part of any member interview is, “What type of person do you think could benefit from using Levels?” By asking them to describe who they think our target audience is, they often describe themselves in greater detail than if we had just asked them that directly. Plus, it helps us get more nuanced, qualitative information about our members that demographics surveys alone usually can’t capture.
Ask Open-Ended Questions — or Let Your Customer Do the Asking
Customer interviewing 101 says to make sure you’re asking open-ended questions — questions that can’t be answered with a simple “yes” or “no” — to get as much detail from customers as possible. Subtly adjust how you ask things to focus on questions that start with “why,” “how,” and “what.” For instance, “what do you think about this feature?” will get you more color than “do you like this feature?”
I take things further and spend a good portion of the interview letting the member ask me things: questions they have about the product, features they’re struggling with, curiosities they have about where the company is going in the future. I find it helps get to the meat of what they want out of the product (and perhaps aren’t getting) much more efficiently, and it’s usually pretty straightforward to translate these questions into action steps for product improvements.
Finally, one of the hardest lessons I learned when talking to members is that silence is okay. Taking an extended pause after the other person stops talking can feel awkward, but if you wait long enough, they will typically start to fill in the silence with more thoughts — helping you dig even deeper into what they’re trying to say.
Don’t Take Feedback at Face Value
Speaking of digging deeper, you should always try to understand the why behind what a user is asking for: why are they requesting a particular feature, why they feel you need to add content around a particular topic, why are they frustrated with this specific bit of UI, etc.
Your customer is the expert of what they want or need, but they don’t necessarily know the best way to achieve that — that should be your team’s expertise. So, rather than take a feature request at face value, work to understand the motivation behind that request. Doing this can be as simple as continuing to ask “why?” to every request they give: Why do you want this specific feature? Why do you find this step frustrating? Why do you want more information on this topic? Learning more about the “jobs to be done” framework could also give you tools to help you approach these conversations.
By reporting the motivations instead of the methods back to your team, you can work together to find the actual best path forward.
Find Ways to Organize Feedback
After spending all this valuable time talking to customers, the worst thing you can do is let that feedback get lost. But with the amount of qualitative information you’re collecting, it’s easy for that to happen.
We’ve been mindful of finding valuable ways to organize and report what we learn in member conversations. We have a heavy documentation culture, so I would first take notes from every interaction and publish them in a dedicated Slack channel, tagging anyone on the team who needed to see a particular point. At the end of every week, I wrote a summary detailing common themes.
Lately, we’ve been looking for ways to make this qualitative feedback even more quantitative and actionable. For instance, to keep us from becoming overwhelmed with the number of categories that feedback can fall into, our head of product David Flinner came up with a set of standardized tags based on high-level aspects of the member experience: understand, usable, learn, improve, track, the arc of the journey, effortless, belong, trust, keep at it, identity formation. (We artificially limited the number of major categories to no more than 10 to prevent things from getting out of hand.) Now we’re able to bucket feedback in a more organized and helpful way. We’ve also been experimenting with tools to support these efforts, like Dovetail, which allows us to tag transcripts, see patterns, and track action items more quickly. It’s TBD if we’ll keep using this specific tool. Still, we’ll keep testing and iterating until we find the most efficient and organized way to identify and act on customer needs.
Monthly recap of tag usage in member conversations in Dovetail.
Close the Loop With Customers
Perhaps most importantly, we always communicate back to members what we’re doing with their feedback. People often aren’t inclined to give feedback to companies because they don’t think it will be heard, so we make sure our members understand that we’re listening.
For instance, I once had a few members within a week say that they wish there were some kind of quick-start guide to Levels to help them get over any initial learning curve. I mentioned it to the team, and within 48 hours, we added a scrappy MVP guide to the app. It was a pretty magical moment when I got to email those members and show them that we truly do make decisions based on their input.
Email letting a member know that we took action on their feedback.
You can’t always react this quickly — if you did, you’d just be playing whack-a-mole with customer ideas and never actually move forward on more significant projects. Even when I think we won’t solve a member’s feedback for a while, I never dismiss it. Instead, I focus on how it fits into our larger priorities. Our documentation culture helps here, too, since I always have a clear view into the product roadmap and can explain that we will address their concerns, but just need to get from A to B to C first to do it properly.
Should every early-stage company spend quite as much time talking to users as I did? Probably not. But anyone working in customer success or product development for a startup should be carving out significant time to get direct feedback from the folks using what you’re making — and hopefully, these tips can help you make the most of it.
What Levels does—and why
Inside Levels
