Josh (00:00):
Okay, let’s jump in, on a Friday forum, May 19th, 2023. And our weekly reminder for what this format is intended to do, it’s our synchronous time, so we’re a remote async company. We have a lot of different functions doing a lot of different things all in parallel. And so, it can be somewhat difficult to keep up with what’s going on across the business, and so this is our opportunity to sink on that. Celebrate recent achievements across the business, hear from members and partners directly, hang out a little bit on Friday, and it’s not the main business analytics meeting. It’s not where we do all of our social connection. So always good to realign on that.
(00:37):
Okay, so this week we are … as you all know, we are running a series of test campaigns, which we’re calling Smoke Tests to understand our positioning and price point and how members are perceiving the various product offerings that we have. And so, these Smoke Tests are super invaluable to help dial in our approach and make sure that what we go live with in a GA or a soft GA, makes sense. So we’re running a second smoke test campaign starting next week. We were intending to launch that this week, but we pushed that to next week based on some analytics that we need to improve. And that one we’ll specifically focus on improving positioning and price.
(01:17):
So those are the main takeaways that we’re trying to get to. We’re still pushing for a partial GA, partial general availability on June 5th. So we’re going to roll that out most likely to a percentage of website traffic. Make sure that we’re seeing what we expect to see and then, we would go live to 100%. So those dates are still holding. Our current paid campaign experiments are showing our customer acquisition cost is decreasing, which is a good thing. So this means that it costs us less to convert somebody into a paying customer. This is good and it’s definitely taken some cycles to get to this point. And then, we did a first user referrals test.
(01:51):
You can see some of the user referrals flow that Karen and the team have been shaping. And first user referrals test showed a 2.8X, I believe, improvement over previous, over the control. So really good and yeah, that’s a good indication, we’re going to proceed with that. I think I said conversions on the home page are continuing to climb, so this is also a really good sign. I don’t think we’ve done any optimizations in the last week and yet, we’re still seeing kind of all time highs on the home page conversions specifically. So a lot of sign up work that went into that. And it’s continuing to bear fruit for us. We’ve done a bunch more guides, content production.
(02:30):
Our guides are still going strong, and our production system is … or our machine is still cranking out really great content. So shout out to the entire organization that’s doing that. We also have on the background, some work going on to enable additional fulfillment options for various new products that are coming to market sensors, et cetera. So a lot of production and logistics work on the backside that is making this possible, so we had a first production order for one of the mechanisms that are important to that fulfillment process. I’m being deliberately light on details here because we haven’t gone live with it. So a lot of work going on to enable us to access additional products that are coming to the market.
(03:08):
And then, we’ve got some sequencing and timing work that’s been done for general availability activation. So this is primarily on the marketing and growth side. So there’s going to be a series of activations that will be across all various channels and content efforts and even traditional ads and marketing. So all of that stuff is in this activation plan, which you can go check out if you’re interested. Tons of work going on there and getting the pieces in place before June. On the member support side, so happiness is at 93%. A huge shout-out to the team. It has just been a huge struggle to clear the queue, to catch up on volume and to get us to a point where not only are those numbers where they should be, but also happiness is super high.
(03:46):
I mean, that’s just a really impressive demonstration of what our support team can do. Volume is down 18% as well. So a little breathing room, which is great. And then, we’ve got autonomy, which is that contracting agency. Three agents are going to be jumping into the queue next week, which will be great, should add even more breathing room, and we can sort of focus on process improvement for a little bit instead of just kind of staying above water. And then, Chris did some cool pilots. You should check out his member experience update for the week, looking at Tableau, which is a data visibility tool and showing how we can really split up our data, everything we’re collecting on our analytics channels to better understand our market and their interests by a huge number of demographic splits.
(04:30):
Okay, Eng Org restructure. So I think everyone’s aware of this, but we are shifting from a pods based engineering organization to a teams based organization. So this rolled out this week. This is great because it allows us to change business priorities on the fly as we are prone to do. So, we can stay super agile without having to mix up pods. And so this will be led by engineering managers. And then, we’ve got a ton of work going on for general availability. So continuous onboarding migration that’s complete. We’re improving performance for the guidance feed. We’re deprecating the old user programs, community feed, accountability calendars and trends.
(05:02):
So all the elements that we need for a soft GA are heavily engineering based. So tons of work going on there. We had 60,000 YouTube subscribers. I don’t know where we started the year, but YouTube is continuing to climb for us. We have millions of views, now, 60,000 full-blown subscribers. And I know we were at single digits not so long ago. This is a shot that was called a while back that YouTube would be big for us and it’s continuing to pay off, which is awesome. We had a bunch of whole new level advisor milestones. So Dr. Lustig, Rob, sorry, hit 500,000 club. We’ve got Dom hitting 50,000. David Perlmiller hit two million and Ben Bickman breached 100,000. So these are the little awards we send out for the whole new level advisors when they hit these landmarks.
(05:54):
And let’s see, finally we’ve got a big partner rollout coming as part of general availability. So Casey had her first podcast appearance in a while with Dana Perino and we went live in Mark’s Pick. So Mark Hyman’s newsletter, which led to a huge conversion week. You can also see on the content side, we had a great men’s health spread in UK. Our newsletter, which is really an amazingly … I mean the numbers on this are really shocking, so we had a Sneaky Spikers newsletter that went out and a 74% open rate with a 12% click-through rate, these are essentially unheard of numbers for the scale that we’re talking about, 200,000 people.
(06:32):
So it’s just really demonstrates how dialed the content arm really is and how much attention people are paying to what we’re doing. And I want to draw some attention down here. We’ve got some performance ads and finally, some of what we’re doing to shape our general availability plan, this is not final, but just to give an idea for how we’re trying to provide optionality while informing people and providing sort of a range of price points. So testing sort of app only versus a part-time use case where you would have a sensor one per month or something along those lines. And then, full-time use and allowing people to sort of select based on their needs, both in terms of data but also price point.
(07:10):
So this is looking really great and there’s just a ton of work. I mean, it’s really hard to take all this complexity and make it simple and intuitive for people. Okay. I think that’s the majority of it. And with that, I want to welcome Dr. Molly Maloof. Molly, it’s great to see you. It’s been a while. Molly was our very first adviser on the medical team. And she’s a partner to Levels. She’s really been raising awareness about the potential for biological observability or the ability to have closed feedback loops from sensors like CGM for a very long time. She’s a Healthspan MD, educator, lecturer at Stanford, multi-time founder. And she recently wrote the book, The Spark Factor and is founder and CEO at Adamo Bioscience.
(07:52):
So Molly, I know you have also a ton of other work you’re doing, including online courses to help people get control of their health and metabolic health specifically. Anyway, it’s been really awesome to work with you for a long time. It’s great to have you back on the forum and would love to hear what you’re excited about in the world of metabolic health.
Dr. Molly Maloof (08:08):
Yeah. My gosh, so great to be here everybody. Let’s see. It feels weird. Am I supposed to be on … I’m not seeing my face, am I supposed to be in front of everyone or it’s up to you.
Josh (08:20):
We can all see you. Yeah.
Dr. Molly Maloof (08:20):
Okay, cool. All right. Weird. So I wrote the book, The Spark Factor, based off of my medical practice, and my work with startups including Levels. I started using CGM very early in 2014. And I remember thinking, this is really important and I don’t know why more doctors aren’t using this. And then, I taught at Stanford a course on Healthspan and then moved to Austin and then turned that course into an online course. And really, the book that I wrote is amalgamation of all these facets of my career. I’m wanting to teach people that underneath cancer or heart disease, diabetes, dementia, not all cancers, but a lot of them is poor metabolic health, is insulin resistance.
(09:08):
And insulin resistance is behind so many different conditions that people don’t realize, and it’s mind-blowing that a lot of doctors are still practicing this sort of outdated symptoms-based drug and surgery based treatment protocols for conditions that are so deeply rooted in energy imbalance. And I really wanted to understand why are people getting sick and how do people get well? So I wrote this book, the Spark Factor, and it’s largely based on mitochondrial health and really teaching people about how does movement improve metabolic health? How do you master your metabolic health with wearable technologies like CGM?
(09:51):
How do you measure metabolic health and what does it mean to measure metabolic health? And then, how does stress play a role in this? And this is a really big one for a lot of people. Everyone thinks it’s just food and exercise, but stress is such a huge part of insulin resistance. And then, there’s the environment and toxicity. So the book is … I think my book plus, if you guys haven’t read this book by Christopher Palmer yet, his book called Brain Energy, basically both of our books are explaining that the majority, 80% of the diseases that we’re seeing in this country are preventable and are into metabolic health, including a lot of mental illness.
(10:28):
So I’m really personally fascinated right now on how, the body is basically your hardware and your mind is your software. And so, I am doing a lot more work with my clients on mindset and stress management and really getting people to use their mind to change their physiology, rather than the other way around. So I’m really curious about where you guys are going with your new products and I want to test out your new products, and I’m just stoked that you guys have been able to have such incredible growth and a huge following.
Josh (11:00):
Awesome. Well, I love the analogy there of hardware and software because we’re experimenting with the same sort of feedback loop where you need both. The hardware is essentially a means to an end. And the experience that we all have is, it’s made better in our software by quality hardware. So I think that the same thing goes for a good product.
Dr. Molly Maloof (11:21):
Totally.
Josh (11:23):
Yeah. I think Caitlin just mentioned the two books you mentioned, Spark Factor and Brain Energy, were part of our first science book club here at Levels, so we follow this stuff very closely and definitely … yeah, yeah, we love all of it. Molly, I know there’s two hours of content we could produce here on Friday Forum on Friday morning, but I think … I’d love to just kind of focus on where you want us to focus. What is something that you would like Levels to do better or differently or start thinking about that maybe you haven’t seen from us just yet?
Dr. Molly Maloof (11:54):
I personally don’t think that enough people are aware of how … so there’s this program called … there’s this program called The Faster Way to Fat Loss by Amanda Tress. And it’s literally intermittent fasting, carb cycling, and basically, it’s like timing your exercise to your metabolic needs for the day. So it’s really employing weightlifting and a bit intermittent, a little bit of HIIT training. And it’s honestly, a pretty good reflection of what my book recommends, which is people need to build metabolic flexibility and they need to learn how to do that. So I think that people really buy based on emotions and they buy based on superficial reasons, right?
(12:43):
So they buy based on fear and then they buy based on how do they want to look. Most people don’t really understand or even really want to care about metabolic health, which is really sad. So I think that it would be interesting to have … I think people are looking for, well, just tell me what I need to eat? What do I actually need to eat here? And I think that you can pretty much give people recommendations based on their … I think that there’s a lot that you can do with CGM to sort of prescribe diets that can help people train their metabolic flexibility more effectively. And the reason why I recommended the faster way to fat loss is because, it’s a really effective, very simple program.
(13:21):
So I’m just kind of curious, what is the promise that you guys are going to be giving people with this new product? Because that’s really what people want to know is what are you promising me as a company? What are you guys going to actually offer me? What am I going to get from doing this? And I think that one of the downsides of sadly optimizing health is that not everybody … not everybody is thinking about their longevity like they should, like the biohackers are … but the vast majority of people are thinking, is this going to help me get paid, get later, or help me lose weight? So, I think that the weight loss path is something that I’ve always wanted to see you guys really work on, more concertedly, and I think it’s something that people will always buy.
(14:04):
I do think people are catching on to metabolic health, but I think most people are still pursuing aesthetics. And that’s one of the best benefits you get from better metabolic health is you just get a better weight. So I would be curious about that. And then, I don’t know where the government and where the legality is in this, but I think most people need to know if they have pre-diabetes or diabetes and they need to know if they have problems with fasting blood sugar or post-peritoneal blood sugar. And I think that everybody should be able to do a home glucose tolerance test and/or a home test that can help you delineate if you actually have metabolic disease.
(14:41):
So I don’t know if that’s something that you need to get approval for, but I just don’t think a lot of people know that they have a problem. And I think that the metabolic score that you guys offer is good, but I don’t think it translates to real-world data in terms of do I have, diabetes? Do I have a metabolic problem? There’s not a connection there. And that’s missing for me. In the app is how do you connect this to real-world problems that people have and/or give people the kind of things that they’re looking to, to get from improving metabolic health.
Josh (15:15):
Yeah. These are really great points, specifically on the needs of the individual, like what people care most about. You’re absolutely right. I mean, there are people who indicate that they want to avoid disease, which is kind of a generic statement, but if you get more specific than that, they tend not to care about the illnesses per se, that are closely tied to their qualitative needs, like you said, weight loss, et cetera. So weight loss is a huge interest of people that use Levels. There’s a huge opportunity for us to really double down there and help people … we’ve done the weight loss challenge or the wearable challenge for a long time, which has had great results where a CGM is the only real accountability layer.
(15:52):
So there’s a ton of opportunity there to show people that metabolic health, improving the hardware, improves the software, like you said. On the second piece of that, it’s a really good reminder that this long term, our objective with the company is to improve health. And the best way to get people’s attention is to show them that there is a problem that they need to resolve, to achieve qualitative improvements, but also, to improve quantitative risk, long term. And it’s a little bit tricky for sure, and I know Zach on the call could probably jump in and share some thinking on exactly how far we can go there, but I think, I’m excited to continue to make the tool as useful as possible, to give people real-time health status and help them understand exactly what that means.
(16:36):
And whether or not we can use specific diagnostic labels anytime soon is going to be a bit tricky with a scaled, sort of, software-driven product, but long-term, I know that that is the future of the space, is that we can use the capabilities of technology to massively improve people’s awareness of whether they are at risk or are currently experiencing something like prediabetes or metabolic syndrome. So it’s a great reminder. We don’t want to just follow the lead of previous products. We have to continue to innovate in the space and help the space adapt to what new software can do.
Dr. Molly Maloof (17:12):
Yeah.
Josh (17:12):
Yeah. Okay. Well, we got a packed meeting. I really appreciate you joining Molly, and please feel free to stick around and check out the rest of what we do on Friday Forum, but I know you’re super busy, so if you can’t, I just want to say thank you from the whole team. I really appreciate you spending some time with us this Friday. And also just generally being such an awesome partner at to Levels over the past going on, it’s got to be close to four years, like three plus years that you’ve been in our orbit. Vice versa.
Dr. Molly Maloof (17:39):
Well, I’m one of your biggest fans and I want to see this succeed. And so I’m just really excited about where you guys are taking this.
Josh (17:47):
Awesome, thanks a lot, Molly. Looking forward to following everything that you’re doing in the space as well. All right, I’ll jump ahead to culture and kudos here. So reminder to finish up performance reviews, so we’ve got the performance review season well underway and I think today is the target date for all individual reviews submitted. The process continues as we do calibrations, and then final wrap-up. So Nicole jump in if I’ve screwed that up, but I think that’s roughly where we are. And I just want to highlight Juan who’s been … who’s really crushing it on a major upgrade to the guidance logic. This is a key component of course, of preparations for general availability and that migration is complete now.
(18:32):
We’ve got a lot of room to play with content delivery in a performant way, which is huge. So a lot of this stuff is really hard to understand if you’re just looking at the UI, like exactly what goes into making that experience better or to adding a specific feature set. So just want to make sure we all can see behind the curtain and appreciate the work Juan has been doing for us. All right, with that I’m going to hand it over to Stoddy for an aside on transparency.
Stoddy (18:58):
Yeah, so I guess in the theme of performance reviews, I’m going to kind of take an angle at this that looks at specifically transparency and performance. So I’m going to quickly go through what does that mean? Where are we coming from when we’re trying to be transparent in performance? A quick example and then why it all matters. So the building blocks of transparency and performance are a lot of the things that have been presented on in the last couple of weeks. Team not a family, treating people like adults. Sunlight is the best disinfectant, feedback is a gift and assume best intent. To be transparent in performance, we really have to nail all of these five building blocks to really be able to fully tackle this one head on.
(19:44):
And to be transparent in performance, we aim to share performance from our team members, across the team from the bottom of the org chart all the way to the top and create trust in this async environment, grow as individuals and teammates and increase our feedback cadence. So how does this fit into our Levels culture? So being transparent in performance helps build trust between team members and highlights that everyone here at Levels is receiving and responding to feedback. It will help team members understand what areas are others struggling with, what areas are others interested in developing. For example, if Matt doesn’t know that I’m struggling with coding and Python, it’s really hard for him to reach out and lend me some of his magic methods.
(20:37):
So this, being transparent in performance takes pressure off managers, to connect people with others on the team to help them with their areas in growth and makes everyone here at Levels and owner in their performance process. So a couple of the assumptions here in my mental model when thinking about transparency and performance is that weaknesses are not character flaws but areas for growth. So each one of us from the top to the bottom has areas we can improve on, focus on or understand at a deeper level. And an inability to see areas of growth or improvement should be a red flag because it either means that you haven’t done the work to reflect on your output or process or it means you aren’t being challenged enough in your role and you should seek opportunities that are harder.
(21:24):
And so the other sort of assumption is that an opt-in approach for this, creates bias and asymmetry. So I’m a strong advocate for making sure that everyone is included in any transparency efforts. And another assumption is that we all want to learn and grow here as coworkers and as people here at Levels. So lastly, I think the last assumption is that growing in a professional role requires reaching beyond what you’re currently capable of and this means that you will likely fail and that’s totally expected and okay. So a quick example from my college fourth experience was when we had set plays, often you’d be sitting there on the sidelines and feel like you understood the play, you’re like sitting there watching it all go down.
(22:20):
And it isn’t until you’re put on the field where you’re running the actual step play and you end up on the wrong spot that you realize like, “Oh, I actually didn’t understand this.” So the feedback cycles and the way that other teammates are able to basically be like, “Hey, this is where you need to be on the field,” it’s way more helpful than just being at the end of an explanation that the coach asks, “Hey, is there any questions?” And no one raises their hand. So this matters in summary because celebrating wins across the org can be hard, and so it can be really easy to not know what people are working on. So added transparency helps people maintain alignment and understand like, “Oh hey, what is Zach working on over there?”
(23:05):
Because it’s like we are growing as a team and so it can be hard. So with added transparency, everyone knows where everyone else stands, showing in the open that everyone at Levels is accepting feedback, learning and evolving and ensuring that we are roadmapping individual improvement in public for everyone. So thanks and I hope the rest of the performance review season goes well.
Josh (23:32):
Great stuff. Thanks, Stoddy. Yeah, I mean, this is one of those things that’s really cool to see because this sort of emergent phenomenon of people being so comfortable even sharing, performance information in an environment like ours, it’s not standard. It’s totally taboo in many companies where people have to interface with each other and there’s sort of a posturing that prevents any real true adaptation and learning. So although we still … this is entirely opt in right now with our performance reviews, people can share if they’d like or not. I do love to see the leaning in here and the pressure for all of us to question why we would hold back in sharing our performance information. So yeah, lots there. Thanks a lot Stoddy. Appreciate it.
(24:21):
All right, main thing, Levels shows you how food affects your health. We all know that this is a piece of it but not the whole thing. And as Molly just said, we also help to highlight how people’s stress impacts their health and how their exercise impacts their health and sleep, et cetera. So kind of in a state of evolution right now and the main thing, but the point is we’re a tool that helps people understand the state of their health and we’re all working towards that. So no change there. Beta numbers to hit. Sam is at a conference right now, but kind of the summary is that we are pushing towards soft GA. We’re going to be getting good information on engagement retention continuously as we trend to the end of beta two.
(25:03):
Soft GA is June 5th, so that target is continuing to hold and right now, the target there is going to be 2000 conversions at soft GA. So we’ll start there. And then engagement right now, 69% as of last measurement this past week. And again, retention we will learn about for beta two, just prior to soft GA because that is a kind of conversion from beta two into general availability. So more to come, the point is we’re holding the course right now and heading into soft general availability. All right, got a content update. I believe this is Tony.
Tony (25:43):
Thanks Josh. Everyone hearing me okay? Cool. Thanks. Yeah, you could go to the next slide, Josh. Awesome. First off, I have to say, it’s been extremely long time since my last update on the Forum. I think it was November or December I saw when I was prepping this slide, which I can’t believe and so much has changed with multimedia since then. And I think one of the great things is I’ve been so fortunate to work with so many of you and just cross-functionally, and I think that’s been one of the most amazing things the last few months. And I really couldn’t have done this without the help of Athena obviously. And our freelance editors and myself we’re all just around the clock just pushing content out.
(26:33):
So it really is a team effort. So I definitely want to mention that and really have to shout out to all of them, because it wouldn’t happen without everyone. I’m not going to dig into all things multimedia for today. I’m really just going to cover just the podcast and the YouTube growth over the last few months since my last update. I’ll cover also some experiments we’ve been running in the last few months and what we’re going to be focusing on as we go into GA. So yeah, you could hit the next slide, Josh. Cool. Yeah, so this is really where we are today right now. These are our latest metrics. I really use the YouTube channel and the podcast interchangeably here because the YouTube channel is just a huge part of the success of the podcast.
(27:22):
It’s like really a chicken and an egg situation here of which one came first and sometimes, I just wonder which one is an extension of which just because looking at those YouTube numbers, it’s just astounding and it really is an extension of the podcast and it’s really helped us increase our reach for the podcast. So I’m going to show some of the experiments that helped us get us where we are today. You could go to the next slide, Josh. So this was definitely an experiment that was a long time coming. I actually remember one of my first, I think, week memos. I think the checkbox … I have to check off this action item now and I think it’s time for it, but this was definitely a long time coming and it was updating the thumbnails for the YouTube channel and just what kind of increase that was going to bring us.
(28:15):
So we finally got into this the last few months and thumbnails are definitely one of those factors that it’s going to help a video succeed for sure. And it was definitely apparent here. For some context, Victor was actually the one that helped me. He made the initial designs for these thumbnails. I AB tested them with our audience. And then, once we got the final design treatment, I published it for Rob’s video, what you see here with Casey and Rob. And it immediately exploded. The next … starting the next set, you’ll see the inflection point right there in that thumbnail, which was amazing. And then once this was happening, I immediately sent over the design to one of our new thumbnail designer.
(29:04):
And just had to mimic the treatment for all of our top thumbnails. And I’m going to show that in a few minutes. That video, in particular, the Robin Casey one, had almost a 300% increase in viewership compared to the three months before. So it was really significant. And then, the other thing I want to cover is what you see here on the left with Austin McGuffie. So I also updated his thumbnails. I had the thumbnail designer. Austin made a bunch of thumbnails in the past and I had him mimic his treatment just so it’s still in line with Austin’s style of thumbnails that he had in the past. And just by swapping that thumbnail, it completely went up the ranks on YouTube. It went up 46 spots, you’ll see there all the way to one, the number one spot for metabolic health.
(29:54):
I think I checked the other day and it’s still there in the number one spot, which is pretty awesome, obviously, these move up and down every week. And even though this isn’t a video … this particular video I think only has like 7,000 views. So it doesn’t really always have to do with the viewership, but it really does improve the rankings, these thumbnails. So I’m really glad that we started this experiment. And then really, the next step … and Mike Haney and I have been talking a lot about this, is just being a little bit more proactive in the approach for this, where it’s just not looking at the keyword, seeing what we’re ranking for, but what keywords do we want to rank for, and then finding out what kind of content we can create for those keywords.
(30:39):
So you could go to the next slide. Yeah, so this was, as I mentioned in the previous slide about once I gave the designer the treatment for the thumbnail, he provided us all of these for all our top performing episodes. And if we just look at just the organic traffic, because we also had paid ads as well, running in the last few months with YouTube ads. But if we just look at the organic traffic, it increased over 2X in views in the last three months compared to the three months before. I think just in the last three months alone, we’ve reached over 1.4 million views since that time. So the thumbnails were definitely a big part of this.
(31:22):
Obviously, we had also some compelling episodes that were also a big part of it, and I’m going to get into that in a minute or so too, but it was just incredible seeing these results. Next slide. So we also ran an experiment recently. You probably already noticed it if you subscribe to the podcast, if you don’t, please do. But we started an experiment on the audio side and our podcast audience is actually very different than it was a year ago. We increased our audience back in January, February with a newsletter that we released. And ever since that time, we’ve definitely developed a whole new audience for the whole new level podcast. So we definitely wanted to lean into that.
(32:08):
And we started repurposing some of our top performing episodes, because this audience probably, they haven’t heard these episodes probably yet. So I started comparing some of these top performing episodes recently. I think we only released two so far, but it was already a 60% increase compared to the original, which is not surprising, but I think the surprising factor was how is 20% increase over actual new episodes, as well? So these are very compelling episodes. Our audience and our new audience really enjoys them. So this is definitely an experiment. We’re going to continue running with repurposing episodes because they definitely haven’t heard these before.
(32:53):
So yeah, next slide. So what are some new concepts that we’re going to be running with the podcast and YouTube? So you probably saw this as well. This was one of our latest ones with Rob and Dom. This was really them, taking the lead on an episode themselves. It didn’t involve any Levels team members, kind of leading the direction as a host. They’re acting as guests and hosts at the same time. We want the audience to feel like a fly on the wall during this conversation. And this was actually very successful just in the last month. It was over 100,000 views. And being able to tie that with insulin resistance and tying these guest hosts with keywords is definitely going to be critical moving forward as we make more of these.
(33:43):
This was … I think, our highest performing episode since last May’s episode with Casey and Dr. Perlmutter for the uric acid episode. So we’re definitely going to be running more of these episodes with advisors and tying these keywords as well within the episode itself. And then, Rob also actually just recorded two episodes last week, and you’ll definitely check that out. I think the next one is coming out next Thursday, so be on the lookout for that one as well. Next slide. Yep, so in the end, why is all of this really important? In the end, we definitely want to convert all of these eyeballs to clicks and then hopefully conversions, especially as we get into GA in the coming months.
(34:32):
So thanks to Paul and the Athena team, we were actually able to … they actually redid all of the clickable links that you’ll find on all of our podcasts and all of our YouTube videos. I think, I don’t know, it’s anywhere between 500 to 1000, let’s say, videos and episodes where we have these clickable links. We always had these, but now, we’re able to track them through post hog and be able to see all this traffic in there. And this is going to be super helpful as we get into GA as well. The other experiment we’re going to be running in the coming weeks is for the first time ever, we’re going to be incorporating a call to action within the podcast episode itself.
(35:19):
So Ben is going to be recording an intro at the beginning of the episode. You could see there’s a short brief sentence there, a one-liner. It’s probably not going to be exactly like that, but that’s going to be the gist of it for the beginning of the episode. So we’re going to start hearing some of these call to actions in our episodes, to help bring awareness to the membership page as well. So this is pretty much a lot of the experiments we’ve been running. It’s been really helpful and it’s been really amazing to see a lot of the success here. And I’m really looking forward to the next few months as we see just the turnout of all of this.
Josh (35:59):
So cool. Tony is our own internal Mr. Beast, just totally gamifying the YouTube algorithm. It’s awesome. And also, I think really important, the content is amazing. So it merits these clicks, and so having to use these tools like better Thumbnails to get people’s eyes on it is just the first step. And then, they stay for the content, which is awesome. All right, thanks a lot, Tony and the entire team. Azure.
Azure (36:26):
Hey. Yeah. So today, I wanted to share some numerical and qualitative feedback on our guidance and then, share some context on the people that we’re talking to. So we can skip on to the next one. So first, I think when you go and you wade through the numbers, it can look a little bit like sifting through these piles of numerical kid toys or letter kid toys, but really, this just reflects that we have a ton of tools in our toolbox right now, to assess who’s engaging, what our demographics are, how people are looking at videos, are different parts in the app. And before we get into it, I wanted to first say that we have basically two key member personas that I think we’re going to learn a lot about, at this.
(37:10):
So there’s Maureen, which is that woman who is approaching or working through menopause. And then, there’s Kevin, who is that younger man who is realizing that his earlier lifestyle isn’t quite working anymore. And these are people we’ve been serving in highly variable proportions throughout our betas and through the Levels’ classic service. And I think the difference in their needs are the crux of our challenges, and what we’re working to address right now. So as Molly said, the get paid, get laid, lose weight, I would add maybe take care of my spouse and kids to that, but our content is definitely trying to speak to those different outcomes all at once. And I think we’re solidly in the camp where the lose weight and take care of your family approach is who we want to speak to and who we resonate with the most.
(37:54):
All right, so we’ve made this lovely introductory program trying to suit everyone and now, we’re talking more about personalization. All right, so onto the next one. So we’ve gotten the feedback from everyone who is writing in that guides that address different topics and maybe focus on a topic are most engaging. So right now, you may even have seen a couple of faces that you didn’t expect to see back in beta two. We have Lynette focusing on CGM. We have Mike focusing on movement. Sonja as our core guide, who I think of as welcoming you into this long-term journey and overall, providing lifestyle support and tips. We have Austin who we just tried out over the last few days and who has gotten a lot of really positive feedback, with a focus on family.
(38:40):
And really kind of just a different fun tone and video style and then, Stacie returning in to focus on cooking. All right, next up. All right, so I want to focus on that guidance tab and first, talk about who is really participating and how has that changed, because I think it provides a lot of context for the numbers. So we know from 2022 just over half of our members were female. And when we launched our Alphas in early experiments that jumped over up to I believe over 90%. Sometimes I think these cohorts were entirely women in those early groups. In beta, the proportion was about 86, so still vast majority women who were responsive and interested in the product. And then, in beta two, we brought our demographics back a little bit more towards what they were in levels classic.
(39:33):
So we have 64% female and my guess is that when we open things up in June, we’ll be even more towards this balance where we have to serve both men and female interests. All right, next up. So because we’ve had that predominantly female feedback for the last several months, I think that’s influenced the direction of the product and content mostly to serve that group. And if we look at not just who has participated but who have we learned from and engaged most with, that little has been predominantly morning. So in surveys, the estimate so far we don’t always have this information is that our responses are over 90% female. In logging, we know consistently from the past that women and then older adults log anywhere between a 125, 135% to activity and logging at about 2X the rate of men and younger adults.
(40:33):
So we have a big discrepancy in who we’re hearing from and how often they are interacting. And given that, here’s the kind of feedback that we’re getting so far. It’s on the next one. All right, so I know this is a little small. Zoom in if you want to see the full time series in detail, but there are a couple of things that I want to call out on habit change. So basically once we engage a member, it seems like they have mostly very positive feedback about what the program is doing for them. So we ask the question each time, these are like your skills out five with five being a strongly agree, A one would be a strongly disagree, that our members who respond say that they agree with on an average of a 4.3 with, I agree I’m making changes to my diet.
(41:22):
Making changes to my exercise comes at a four on average. And this program is helping me stay accountable to myself, and my goals comes in at a 4.4. That tends to peak at about week two for our betas, but it overall stays pretty high. And we’re seeing that same kind of pattern emerge now in beta two as compared to beta one. Timing also seems to matter for habits. So overall, we’ve received a positive response averaging of four, 4.6 for morning routines to this idea that you could have a theme to your content stream that would allow you to progressively build habits based on your time of day routine, starting with what you would do when you first wake up in the morning and progressing through the rest of your day.
(42:06):
And then finally, that especially without a CGM, guidance seems to be crucial for Maureen. So we did the whole experiment because we wanted to see, could we keep people interested in making progress with the Levels app during the time that they didn’t have that continuous glucose data. And it does seem that indeed when people rate the different parts of the app that they prefer, especially without a CGM, guidance tab comes in on top. All right, next one. So I think that feedback is super helpful, even though it is quite biased towards Maureen and I wanted to share a couple of quotes on what Maureen is asking for topic wise, what she says about the types or type of guide that she’s most looking for. And then the same thing for Kevin, our male person on the bottom.
(43:00):
So first on topics, we’ve had over 100 women write in so far and in surveys over 90% saying that they would like more content that is specific to aging and women’s metabolic health. And then, behind that are topics that are most requested, are more recipes which we’re addressing and then these family tips. So I think that’s something that is consistent and has been consistent for the last few months. Whereas with Kevin, it’s a little bit more on the logging experience, more technical advice. We’re getting a lot of these guys who follow Rob, follow Peter Attia and who want to be able to dive in and talk to an expert. They’re a little more interested in frequent CGM use, seeing more personal data like personal spikers and in understanding our scoring.
(43:47):
And then on the guidance side, I really love this quote, it’s just one person obviously, but I think it captures the sentiment from a lot of Maureen’s writing in so far, she’s looking for someone realistic, attractive and relatable. She loves our current people, but she really wants to also see a woman who is 50 plus inspiring and attainable. And then, for our Kevin side, they seem to be a little bit more agnostic about exactly who they hear from. They’re looking for that 201 level advice, maybe more experts, but they don’t mind as much, if it’s delivered from a man or a woman. And they do so far seem to really like our test out with Austin McGuffie. All right, next one. Okay, this slide has a lot of words, but I think they’re relevant, so bear with me.
(44:33):
I recently rewatched the graduate and I know this guy is a little bit younger than our ideal Kevin persona, but it really made me think this is a very stark contrast between the needs, the worldview, the comprehension level of our younger Kevin’s versus our older Maureen’s. So in this compare and contrast, the top half is a little bit more behavioral. Yeah, Rob says we need an Elaine. Totally agree. The bottom half is a little more on needs. So Maureen is really into social accountability. She speaks help from people. She writes to support when there’s a problem with renewal. She often tends to think that things are her fault if they go wrong. She’s more patient with us, even if she’s a little bit more tech-savvy and needs some help onboarding.
(45:20):
Whereas, Kevin, he’s a little bit more into numerical accountability. He’s a little bit less tech-savvy … or sorry, a little bit less patient but more tech-savvy and he doesn’t reach out as much to us behaviorally and needs wise, Maureen cooks more for her family. She has actually experienced some pretty rapid waist growth and weight gain as opposed to that slower getting out shape over the 20s and 30s that these guys are experiencing. She’s got poor sleep, she’s going through menopause. She’s a little bit more introductory when it comes to strength work, versus Kevin who knows what it’s like to be fit, knows a little bit more about strength and maybe just needs to get back on the wagon. And I think what they share and is this middle space that we’ve been mostly occupying is that they want something more personalized.
(46:08):
They want a CGM on demand when they want it, they want the material at their level and they want us to help them deal with weight and fatigue that they’re experiencing as a result of poor metabolic health. So it’s a lot for us to handle and I want to call out those needs just because we’re mostly hearing from that Maureen, but we have a pretty good idea of what Kevin is looking for too. So next one, a little bit of how we’re going to address this moving forward. As you saw, we are working to expand the number of coaches and topics that we’re offering to boost content on aging and women’s health, offering a variety of strategies to hopefully help people choose their own difficulty level or entry point into our programs.
(46:50):
And then, accountability and action, you’ve heard from those in forum over the last couple of weeks, but I have one more slide on personal goals and logging and then, some love for the program. So a little bit of a sneak of what’s next. Thank you Alan and Victor. So the metabolic checklist is something that we’ve received a lot of feedback that people like the idea of this checklist, but they want it to be a little bit more actionable. So we’re bringing in fasting, we’re bringing in some progressive goals and working to pull that checklist more into the overall app experience as opposed to a standalone thing in the hope that this will be more engaging to that, that Kevin side of the equation. Okay, and then, I think we just have one more.
(47:35):
All right, so a little bit of love. As you can see, I’m very text heavy today. Across beta one and beta two, we’ve gotten a lot of specific love for the program. People saying that this version is so much better even though we’re working really fast and we’re kind of scrappy, people are noticing the difference. They love to start their day with the guidance videos. This one in the middle, the videos are phenomenal, unlike anything I’ve seen anywhere else and they’re very relatable that just came in last night about beta two. People saying that they’ve learned so much, they love our weekly themes and that they’re really liking this much more personalized approach.
(48:10):
So I think we’re on the right track. I think we’re trying to solve a very hard kind of two-part problem right now and we’re throwing a ton of really good work at it. So also extra thanks because these numbers are coming from lots of work on the backend for Eng to improve our video and app engagement stats to all of customize work to our XPs working pretty much around the track to keep these up to date. And then to everybody else on the content team. Thanks guys,
Josh (48:39):
Really nice dive. Thanks a lot, Azure and the rest of the team as mentioned. This is a super cool insight. I’m going to have to rewatch this section again because there’s a lot in there to unpack. Awesome. Okay, quick hiring update. We have an opening for backend software engineering and otherwise, if you or someone you know is interested, check out levels.link/careers, for more info and general application. And then, we got some time for individual contributions. So I’ll stop the share here and recommend throwing your hand up if you have the reactions button, if you want to share something with the team. Azure beat me to it. Go for it.
Azure (49:25):
Hey, well, this is a little bit silly because it’s being resized right now, but I am super happy personally because I got engaged a little bit over a week ago. So I told a few people but wanted to share here too. So super excited and I feel like right now, both personal life and levels are at this very high pitch of change, exciting, I don’t know what’s going to happen, but I feel excited. So happy about that.
Josh (49:55):
Congratulations. That’s huge. Big life changes are great. Well, I don’t have anything as exciting as that, so I am moving this weekend and that’s about it, which I’m personally am professionally excited for. Those two things have emerged now in my life and yeah, generally, just I think continuing to feel a lot of optimism about the pieces, the raw materials that we have. And it’s exciting to see things like these nonlinearities, like what Tony just shared on YouTube for example. We have such excellent raw material and by changing a few of the pieces, even just thumbnails, just to put it on people’s radar, it really resonates so well. And I feel like it’s kind of generally what we have here at Levels.
(50:39):
Just a lot of really exceptionally high quality stuff that we can improve people’s metabolic health with, once we figure out the sort of combination lock. So yeah, not very specific, but kind of how my current vibe feels. Chris.
Chris (50:57):
Super excited about the upcoming GA and all the work. So thank you Azure, that was a great update and from the things coming. It’s super exciting to get that in the hands of more users and get feedback, and continue to learn. So just again, thank you for everyone that’s been touching the new version, which essentially is everyone in the company. On a personal side, last week I shared that I was excited about six hours mowing, in my farm, talk, update, farm, talk, update. My tractor broke down twice, a wheel fell off, the deck started smoking. So I did what any recent California to Montana move person would do. I brought out all my tools and I set them on the bench and I went to the store to buy a new tractor. I can’t fix it, just give it to the in-laws and go get yourself another one.
(51:51):
Yeah, so serious upgrade on farm life and having fun driving around at 10 miles an hour, have seen how fast this thing can go.
Josh (52:02):
Chris, if you haven’t watched Clarkson’s Farm on Amazon Prime, I highly recommend it.
Chris (52:07):
Yeah.
Josh (52:08):
Okay.
Chris (52:09):
Yes. That is my life, just at lower scale.
Josh (52:12):
So you bought the Lamborghini tractor then?
Chris (52:14):
No, that lower scale.
Josh (52:17):
Very nice. Miz.
Miz (52:20):
Always entertaining. Chris, thank you. On the work side, I had the chance to do a user feedback call this week along with Hwei. We spoke with two members. I watched Maza’s call from last week I think, and then, had mine scheduled this week. So it’s always really, really fulfilling talking to these members and hearing their experience, understanding just the depth of their knowledge on these topics is fascinating to me. We focused a lot on cost and price and how often folks are using it, why they’re coming back to it every other month or once every three months or whatever it was. So a lot of really, really good insights from that.
(52:58):
And Scott has made great progress this week, so we’re on schedule and it’s been really nice that, just kind of working with him. So shout out to Scott for pushing forward and taking a ton of ownership. On the personal side, I got an Apple Watch Ultra this week after my series seven battery started to go slow. So I saw I think mics and mats and whatever when we were in Austin. It’s the size of a 1999 computer monitor display on my arm, but the battery life is phenomenal and it is a nice device, so we’ll see if it sticks. Yeah, the gateway that Chris was selling in the 90s, it’s now on my wrist and otherwise, I have a bike race up in Manchester this weekend that I’m not ready for, but super excited. So in it for the fun and yeah, that’s it for me.
Josh (53:44):
Sounds awesome. Tom.
Tom (53:49):
Hey everyone. Professionally, just wanted to give a little bit of a shout-out to the growth team across the board. I think product has very clearly been pushing extremely, extremely hard and I think growth has been maybe just a little bit quieter over the last couple of months, but wanted to just recognize that people have been very literally working around the clock over the last couple of weeks, which we don’t celebrate when people are up super late working, but we do recognize it and appreciate it when it’s necessary at times. And there have just been really fast turnarounds and tight timelines and an ever-changing landscape as we get feedback from the market and prepare for general availability.
(54:38):
And there’s just been an absolute ton of work, going into that and it can be stressful at times and a lot of unforeseen challenges. So a lot happening behind the scenes with growth these days and just hugely appreciate all the work that’s going into it. And honestly, this touches people not even on the growth team. It’s a lot of people that are contributing to this, so that’s awesome to see. Then personally, mostly just preparing for a wedding and bachelor party season over here. So I’ve got something like 13 weddings and bachelor parties just in the second half of the year, and it’s in theory, fun but truly horrifying. That’s what I’m staring down the barrel of, currently.
Josh (55:23):
Don’t be scared. Jump in with both feet, Tom. Rafael.
Rafael (55:28):
Hey everyone. Yeah, so professionally, I’m really excited about this sprint. There were folks on all the performance improvements. I think it has been a long time coming. Yeah, we built a lot of new features very quickly and I think now, it’s time to just take a step back and see where we can improve. So thanks to John and Justin for helping with that as well. Justin actually has some really cool looms where he debugs certain things that we were working with and it’s really great to see someone work at that level of expertise. So very cool. Personally, last night, I went to see The National Life in downtown Chicago in this beautiful 1889 theater. It was great. I had high expectations for their set design, but it was way more than I expected, so it was amazing.
(56:17):
So if they do stop by your city, you should really check it out. It was amazing, just to be in that setup was great. And the music of course, was great. And also, my dog is back. He went missing for about five weeks. I boarded him. When I went to see my family in Brazil. The boarding service lost him and someone found him in the middle of a cornfield, 45 miles south of where he started. So he’s back, he’s healthy and yeah. I’m glad he’s back. So that’s some happy Friday news there. Thanks.
Josh (56:55):
That sounds like … I’m pretty sure there’s a movie about this called Homeward Bound that I saw as a kid. Pretty amazing. Yeah. Glad you got him back. All right. Anyone else want to jump in this last minute of the Friday Forum? All right, cool. Well, great meeting. Thanks everybody for contributing, for all the great work and have an awesome weekend.