#128 – All things in marketing attribution | Josh Mohrer, Paul Barszcz, Ben Grynol

With online sales, all customer touchpoints are important for attribution. But they can be tough to track, because attribution is really a multi-touchpoint game. So how can you determine which touchpoints are giving you the most returns? In this episode, Levels’ Head of Growth Ben Grynol chatted with our head of Global Operations Josh Mohrer and Digital Marketing Manager Paul Barszcz about the importance of marketing attribution, different ways to determine where sales come from, and how to build trust with your customers.

5:26 – How to define attribution

Josh said attribution is a way to get to know a little bit more about your online customers: how they got to your site, where they came from, and why they’re there.

Why don’t we back up a little bit and just define what we’re talking about here when we say attribution? With a business on the internet, whether it’s a website or an app, there’s an opportunity to know more about the customer than you would if you had a little store on the corner. At your store, somebody walks in, it’s not clear to you where they learned about your store, why they’re there, if there was a specific advertisement that you ran that got them in the store. But with the internet, there’s an opportunity to know those things. Either you ran an ad and someone tapped it and went into your app and they bought, or they saw a search ad and they clicked on that and they went to your website. So what we’re talking about here just generally is the idea of knowing where the business is coming from. When someone joins Levels, how did they learn about us? How did they get to us? Is it the result of an effort that we took? And that’s really what we mean by attribution.

8:06 – Attribution is a social science

Paul said the more sources of ad traffic you have, the more difficult it becomes to identify original attribution.

If you’re running a Shopify store, a very simple one selling t-shirts and you’re running Facebook ads and Instagram ads, I think your attribution model is quite simple there if those are your only sources of traffic. If you’re dumping money into Instagram and seeing a sale out of that, you most likely know that the person came from there. Things start to get complicated once you expand your business and start adding different touch points across the way. So once you start adding influencers, let’s say you start putting money into SEO, display ads on different websites, and then like you said, billboards, radio ads, TV, things start to get very complicated. And that’s when I think attribution turns into something that someone told us about. It turns into a social science rather than a hard science where you start making assumptions that your radio ads or your TV ads or your billboard ads are actually working. Of course, there’s some ways to measure it. You could do some incrementality tests around that, but it’s not easy.

10:07 – Figure out where demand is generated

Josh said it’s easy to track the final touchpoint before a purchase, but the real challenge for companies is figuring out where the buyer first got interested in the product.

An attribution system, or any kind of analytics like Google analytics, for example, which we use will only really have knowledge of that last touch or it will highlight that last touch as the important one, because that sort of closed the deal. That was sort of the alley oop and the Google ad slam dunked it. But it doesn’t take into account how the person got interested in the thing. And so the challenge for us is, and really for anyone who’s doing this is to try to figure out where the demand is generated. So for Levels it could be this podcast, it could be one of the interviews that our founders do on others, it could be an article, it could be our blog. There’s all these ways that someone might learn about Levels. Then they hear a special link on a podcast for like Hyman or Sinclair, and they go to that and we say, Sinclair and Hyman generated all this demand for us but the reality is there was a cushion under that that was getting people interested. That’s just what closed the deal.

11:12 – Examine your spending habits

Paul said it’s important to have a macro view of what you’re spending and where you’re spending it to make sure you’re getting a proper amount of ROI.

I think there’s a quote that kind of summarizes what you said that gets passed around in a marketing world quite a lot by John Wanamaker. It’s, “Half of the money I spend on advertising is wasted, the trouble is I don’t know which half.” I think moving forward for Levels and any other business, and I’ve done this in the past this as well, it’s always have a macro view on what you’re spending, where you’re spending it, and see if you’re getting a proper ROI at the end of the day. Of course, there’s ways to evaluate if your channel is performing or not. But at the end of the day, you also have to look at the dollar that you put in, are you getting the dollar back or some multiple of it?

13:30 – Take the long view

Ben said you have to have a long-term view of return rates, because if people buy your product once and aren’t satisfied, in the end that’s not good for your business.

What people don’t realize is maybe, and this is why you have to look at, we’ll just step back and call it capital M Marketing or growth, is this having this wide view and this long lens is what if the return rate’s really high? You got a ton of people to do this thing, to do the behavior that you’re trying to measure, but then because you hadn’t built up enough brand equity through other channels or through educating people, you got them to do the thing, but what’s the long lens on that? If a bunch of people return the thing it’s like, eh, that’s actually not helpful so you kind of needed the Super Bowl ad to build trust. And that’s the loose example, but that’s why this game is so hard and it’s a slippery slope because it takes so much ongoing effort to continue to educate people and gain their trust. Really they’re giving you their trust so that when they convert, then it leads to the only brand touch points that you can measure through brand lift surveys, which are how did you hear about us, word of mouth? You’re never going to get a direct conversion, but it’s good to know if that’s holding a significant portion of the pie as far as like why people built that trust with you.

14:45 – How much will you pay to acquire a new customer?

Josh said you have to determine what you’re willing to pay to get a new customer, because that will directly influence where and how you spend your budget.

Let’s just imagine the JM’s widgets. It’s a great product guys, you should really consider it. But the thing is they only last a month and after a month you need to buy another one. And luckily, most people do that. And so what I do when I’m buying my ads on the Super Bowl and on Google is that I take into account my assumption that you’re going to buy multiple widgets over time and so that lifetime value of all the purchases, I’m going to spend into that. I’m going to go out and say, I’m willing to spend all the money I’m going to make on someone through their lifetime with me, and that’s what I do. And actually, I think that’s kind of a losing strategy, but it’s pretty common. Because thinking about what is my willingness to pay for a new customer? Is it what they spend on the first sale? Is it what they’re going to spend across all sales? Is it something in the middle, maybe the first year? That’s part of the equation because the attribution sort of feeds into that. The more I can attribute to the Google ad, maybe the more I’m willing to spend on it.

22:46 – The importance of an omni-channel approach

Ben said using a single-channel approach means you’re entirely dependent on that avenue of revenue, whereas if you’re omni-channel you have a distributed audience.

What happens when you’ve got a single channel approach for capturing or for harvesting demand versus being omni-channel for other conversions or playing the full funnel game when there’s an outage like Facebook goes out or when the cost of acquiring those customers goes up significantly, all of a sudden people say, oh my gosh, I don’t know what to do. I lost so much revenue. I don’t remember when that Facebook outage was, but there was one relatively recently. I think it was the fall. And it might have been like a day or two days. And Facebook, this is Meta outage, we’re calling it because it was Instagram, Facebook, everything. And everyone who is running that playbook was saying, I don’t know what to do. I’ve lost all my demand. And so that gets into this idea of why omni-channel approaches are so important. And having this idea of almost a conversion portfolio. It’s almost like an investment portfolio where you want to have diversification to make sure that you are hedging against all these different channels, knowing that there’s also different traffic coming in through different sources and starting to think through how to play that game, as opposed to just going deep on a single channel.

26:51 – Make your product worth the buy

Ben said you have to take a holistic view of people buying your product, because if the product itself isn’t worth buying you won’t get a lot of repeat customers.

If people convert and the product isn’t great, then it’s all for naught. It doesn’t matter how great your conversion levers are and your attribution model. The foundation is and now we’re just pontificating a bit, but like build a great product that people love and then all the other aspects of what you’re doing become a lot easier so that when you can have the multiple touch points come with attribution, all the preliminary ones that being word of mouth, that being the initial awareness played through some, whatever, some billboard, some YouTube video someone saw that had nothing to do with the last touchpoint of conversion, but helped to build that trust. It’s a really hard game and so it’s so important to step back and realize that you have to really take this holistic view in building a product, educating people about it, and then getting them to actually convert is one mechanism of this whole game. And it just takes a really long time.

30:38 – Measure the performance of your channels

Paul said you can’t just look at the performance of one or two metrics. You have to look at the channel’s performance as a whole.

If we’re looking at our attribution, we got to make sure that we’re actually measuring the performance of our channels and not only looking at one or two, several metrics where it’s did this person buy or whatever, but also try to look at our impressions and see is the number of impressions that we’re getting, is it really high and is our conversion rate really low? And if it’s really high, you want to ask yourself, are you paying too much for these impressions? Is your CPM price too high or even is it too low such that let’s say you’re paying a dollar per CPM, is that too low and you’re just getting jump traffic? So you just got to, I guess, on a regular basis kind of prune your ad spend, and also make sure you know where your traffic is coming from and set up some filters or ways to kind of clean this up.

36:12 – The Oprah effect

Ben said you can’t rely on a single product placement for success, no matter how famous the person or channel you’re using.

What happened in the late 90s, early 2000s was a lot of these big brands that were consumer brands were doing the Oprah effect. If they could get Oprah aligned with their product and Oprah would do the under the seat giveaway, that was kind of like the market maker for them. That was the holy smokes, look what happened to our company. Well, you can’t really measure that. You just know that generally things are up and to the right for your business because of this thing and it happens over time. Now, fast forward to today, the game that a lot of these DTC and consumer brand companies are playing is the Oprah effect of, it’s mostly the podcast world, but it’s like the Joe Rogan effect, the Tim Ferris effect. It’s if you can get aligned with these companies, let’s throw out the door, well, it’s actually a real scenario where we want to know that there is some attribution associated with the traffic that’s coming from any source that any of the prominent thought leaders that we are aligned with but the game gets a lot different when we’re not offering incentives for people to convert. Once we are post liftoff, that’s not the game we’re going to play. We’re not playing the “and get your free…” We’re playing the hey, this person believes in what we’re doing and they have enough trust in us as a company that they’re saying, try this out. That’s a very different game.

Paul Barszcz ****(00:06):

You can spend money on Facebook, Google, Pinterest, Snapchat, and everywhere. And if someone makes a purchase, let’s say your attributional windows seven days. If someone makes a purchase within those seven days and they saw each one of those ads, let’s say remarketing to everybody on every single one of those channels, each one of these platforms will try to claim that purchase. And in each of these platforms, you’re going to see one sale. And then when you aggregate all of those sales together, you’re going to see like, oh look, I had six sales, because Facebook has one, Google has one and everyone, but then when you look in Shopify or whatever e-commerce platform you’re using, you only see one sale out of that. And so you got to ask yourself which channel actually influence the person to make that purchase?

Ben Grynol (00:58):

I’m Ben Grynol, part of the early startup team here at Levels. We’re building tech that helps people to understand their metabolic health, and this is your front row seat to everything we do. This is a whole new level.

Ben Grynol (01:24):

Where did the sale come from? Well, that is all things attribution. That is, if you’re in the game of sales and marketing, that is something that people consider, especially when it comes to things like eCommerce or digital sales. It’s very different than the attribution or trying to find a path of where a sale may have come from through something like brick and mortar. The game is much different. If you use things like TV ads, if you use things like billboards to bring awareness to a product, to a company, it’s a very different effort than a company that only sells online, where the paths to where a sale might have come from or where traffic might have come from are very different, the outlook is different. But what matters is that all touchpoints are actually important. When you start to think about how people actually become aware of new companies or new products, what are the strongest levers?

Ben Grynol (02:18):

Well, in many cases, word of mouth. If someone, if your friend, if your family tells you about a great product, there’s a higher propensity that you will probably check out that product. And so that is one lever, but that’s also one touchpoint. Does the sale come from that touchpoint? And maybe the sale doesn’t come from word of mouth. Maybe it comes from hearing first from a thought leader or seeing a product or a company advertise somewhere or discussed on your favorite podcast or seen in a publication, maybe a digital media article, maybe you saw a video online about it. Anyway, there are many places where attribution can come from. The important thing is to understand traffic and to understand the strength of these different levers, where this traffic comes from, how it can be measured, and what to do about it, where to play and how to win.

Ben Grynol (03:07):

And so JM, Josh Mohrer, head of global ops, and Paul Barszcz, digital marketing manager at Levels. The three of us sat down and we discussed all things attribution, this outlook on where sales come from, why we measure, and what some of the downfalls can be when you get too deep into attribution. Can it be beneficial or is it a distraction in some cases? We dissected all of these things related to attribution. So if you are into sales and marketing, if you like hearing conversations like this, it’s going to be interesting. If you’re looking for something related to metabolic health, it’s probably not in this conversation, but there’s many more of those to come. Anyway, here’s where we kick things off.

Ben Grynol (03:53):

Okay, let’s dig into this. So we’ve been talking a lot internally about attribution and some of the paid placements. To give some context, to date the only real paid we’ve been doing is placements with podcasters and we’ll say podcasters and influencers and the way that people could get into the beta for the most part was through some link that was custom created for that partner. And so that gave us some sense of where traffic and conversions were coming from, but we haven’t done a lot of the typical things that you would do as far as performance goes like paid social and search, paid search, and a lot of the other paid marketing initiatives that we could do where we could start to analyze some of the analytics behind attribution where conversions are being driven from.

Ben Grynol (04:41):

The caveat to all of this is this conversation around attribution and some of the challenges with it when you start to think of why attribution is really a multi-touch point game, if you want to call it that for companies in some spheres and not others, and why if you don’t have the right lens on it, that being spinning on things like out of home or TV or radio, where attribution’s harder to follow a path, it can still be important for some companies. So why don’t we break down this conversation around attribution and some of the benefits of it, and when it works for some companies and then why some organizations need to think differently about multi touchpoint attribution and how they can think about conversions and what they gain from it.

Josh Mohrer (05:26):

Why don’t we back up a little bit and just define what we’re talking about here when we say attribution? With a business on the internet, whether it’s a website or an app, there’s an opportunity to know more about the customer than you would if you had a little store on the corner. At your store, somebody walks in, it’s not clear to you where they learned about your store, why they’re there, if there was a specific advertisement that you ran that got them in the store. But with the internet, there’s an opportunity to know those things. Either you ran an ad and someone tapped it and went into your app and they bought, or they saw a search ad and they clicked on that and they went to your website. So what we’re talking about here just generally is the idea of like knowing where the business is coming from. When someone joins Levels, how did they learn about us? How did they get to us? Is it the result of an effort that we took? And that’s really what we mean by attribution.

Ben Grynol (06:24):

We’re going to use JM’s Bodega as our fictional brick and mortar.

Josh Mohrer (06:28):

JM’s Bodega on the corner of 71st street does very well. And it’s not my bodega, but there is one.

Josh Mohrer (06:36):

Yeah. And so I think we were talking as we come out of beta ourselves and start to advertise, are looking to grow, where we spend our resources is an important decision for us and we’ll probably do that based on the data that we see through things like attribution.

Ben Grynol (06:56):

To also give context to this. Paul wrote a great memo on analytics and attribution and there’s a funny graphic that we keep referring back to that is the last point conversion was some person on mobile clicked an ad or something, and it showed their user journey of like the 900 other touch points they had beforehand, seeing a billboard, a friend said go get this thing, and then the exec at a company is like dump all the money into whatever that last touch point was, and that is the challenge around attribution.

Ben Grynol (07:26):

So I don’t know. Paul, what are your thoughts? Because when we talked about this earlier, you had a really interesting thought where attribution and understanding traffic from single sources where it’s very clearly measured that being direct to consumer companies that are, we’ll frame it as playing the Facebook game, they just keep dumping money into one channel, it’s actually okay because that’s their game as they’re trying to drive conversions for some like DTC product. Whereas, Safeway is looking at the billboard and the coupon game and all these other out of home initiatives where it’s a lot harder to measure that attribution. And so I don’t know. What are some of your thoughts around it?

Paul Barszcz (08:01):

Yeah. It’s not easy. I think it depends on what stage your business is at. For example, if you’re running a Shopify store, a very simple one selling t-shirts and you’re running Facebook ads and Instagram ads, I think your attribution model is quite simple there if those are your only sources of traffic. If you’re dumping money into Instagram and seeing a sale out of that, you most likely know that the person came from there. Things start to get complicated once you kind of expand your business and start adding different touch points across the way.

Paul Barszcz (08:36):

So once you start adding influencers, let’s say you start putting money into SEO, display ads on different websites, and then like you said, billboards, radio ads, TV things start to get very complicated. And that’s when I think attribution turns into something that someone told us, Alexei, it turns into a social science rather than a hard science where you start making assumptions that your radio ads or your TV ads or your billboard ads are actually working. Of course, there’s some ways to measure it. You could do some incrementality tests around that, but it’s not easy.

Josh Mohrer (09:17):

Let’s do an illustrative example just to get everyone sort of understanding what we’re talking about here. So let’s say JM’s widgets. I just opened a new website. It’s on Shopify, JM’s widgets and it’s great and I’m so excited about it that I buy a Super Bowl ad. And I spend the 6 million bucks on Super Bowl ad or whatever it is. Millions of people see it and they say, oh, JM’s widgets, and then they go and search on Google for JM’s widgets and see my ad there and they click that ad and I get a ton of sales. The last touch attribution or kind of what our analytics systems will say happened is that that Google ad was wildly successful. It doesn’t take into account that there was a Super Bowl ad that generated the demand and the Google ad kind of harvested the demand.

Josh Mohrer (10:05):

So the idea an attribution system, or any kind of analytics like Google analytics, for example, which we use will only really have knowledge kind of that last touch or it will highlight that last touch as the important one, because that sort of closed the deal. That was sort of the alley loop and the Google ad slam dunked it. But it doesn’t take into account how the person got interested in the thing. And so the challenge for us is, and really for anyone who’s doing this is to try to figure out where the demand is generated. So for Levels it could be this podcast, it could be one of the interviews that our founders do on others, it could be an article, it could be our blog. There’s all these ways that someone might learn about Levels. Then they hear a special link on a podcast for like Hyman or Sinclair, and they go to that and we say, Sinclair and Hyman generated all this demand for us but the reality is there was kind of like a cushion under that that was getting people interested. That’s just what closed the deal.

Paul Barszcz (11:10):

Yeah. And I think there’s a quote that kind of summarizes what you said that gets passed around in a marketing world quite a lot by John Wanamaker. It’s half of the money I spend on advertising is wasted, the trouble is I don’t know which half. I think moving forward for Levels and any other business, and I’ve done this in the past this as well, it’s always have a macro view on what you’re spending, where you’re spending it and see if you’re getting a proper ROI at the end of the day. Of course, there’s ways to kind of evaluate if your channel is performing or not. But at the end of the day, you also have to look at the dollar that you put in, are you getting the dollar back or some multiple of it?

Josh Mohrer (11:53):

Yeah, for sure. And it can be tricky because if you only look at the last touch, it might seem more optimistic than it really is. In that Super Bowl example, your ROI on the Google ads is enormous. Sometimes what I’ve seen done is you’ll take all the marketing efforts as kind of the denominator of what’s going on and think about CPA or what you spend to secure a new customer and kind of divide it against all the marketing spend so that you’re not incorrectly attributing to just one thing.

Ben Grynol (12:28):

Yeah. It’s the idea of the funnel, which you alluded to where it’s really easy to trick oneself, to trick an organization into thinking, okay, dump all the money into paid search, because that’s where it happened. That’s where it happened and meanwhile, you have to take a step back and go, okay, this happened because of driving traffic from the awareness standpoint, using the widgets example in the Super Bowl ad. But where it gets really interesting is attribution or measuring conversions can also be a slippery slope too, because if you haven’t built up enough brand equity, let’s assume that JM’s widgets were, it was this product that it was a bright, shiny object. So somebody converted on it because they saw this ad. Let’s say there wasn’t a Super Bowl ad. Let’s say that it’s more along the lines of dumping money into the Facebook engine and you get these conversions.

Ben Grynol (13:20):

Well, that might be the first time that people have heard about this product and they just converted because it looked bright and shiny. And you go great, we got conversions. But what people don’t realize is maybe, and this is why you have to look at, we’ll just like step back and call like capital M marketing or growth, is this having this wide view and this long lens is what if the return rates really high? You got a ton of people to do this thing, to do the behavior that you’re trying to measure, but then because you hadn’t built up enough brand equity through other channels or through educating people, you got them to do the thing, but what’s the long lens on that? If a bunch of people return the thing it’s like, eh, that’s actually not helpful so you kind of needed the Super Bowl ad to build trust.

Ben Grynol (14:08):

And that’s like the loose example, but that’s why this game is so hard and it’s a slippery slope because it takes so much ongoing effort to continue to educate people and gain their trust. Really they’re giving you their trust so that when they convert, then it leads to the only brand touch points that you can measure through brand lift surveys, which are how did you hear about us, word of mouth? You’re never going to get a direct conversion, but it’s good to know if that’s holding a significant portion of the pie as far as like why people built that trust with you.

Josh Mohrer (14:42):

Yeah. And it’s a similar idea. I mean, let’s just imagine the JM’s widgets. It’s a great product guys, you should really consider it. But the thing is they only last a month and after a month you need to buy another one. And luckily, most people do that. And so what I do when I’m buying my ads on the Super Bowl and on Google is that I take into account my assumption that you’re going to buy multiple widgets over time and so that lifetime value of all the purchases, I’m going to spend into that. I’m going to go out and say, I’m willing to spend all the money I’m going to make on someone through their lifetime with me, and that’s what I do.

Josh Mohrer (15:23):

And actually, I think that’s kind of a losing strategy, but it’s pretty common. Because thinking about what is my willingness to pay for a new customer? Is it what they spend on the first sale? Is it what they’re going to spend across all sales? Is it something in the middle, maybe the first year? That’s part of the equation because the attribution sort of feeds into that. The more I can attribute to the Google ad, maybe the more I’m willing to spend on it.

Paul Barszcz (15:49):

You guys mentioned this in the previous podcast with Will Wong, and the question was if you’re an early startup or early business and you don’t know, what’s the lifetime value of your customer, let’s say you’re in business for six months and you don’t know if people are going to stay six months, one year, two years, what metrics should you be looking at? And something that Will Wong mentioned was the payback period. And I think that’s an important metric to look at as well. I think, especially in a CEG world or any type of product where it’s consumables, I think you can also have the lifetime value, which is not always easy to measure or given that your relationship with the company could last for years. But I think the payback period is something that’s quite important to look at to have a sustainable business.

Ben Grynol (16:42):

Yeah, and that’s why the idea of building the trust, building the right level of transparency and education in the process, so your widget looked great, people converted, but they weren’t aware that it had a shelf life. They thought they were buying a piece of hardware. Let’s just use a Nest thermostat. And there’s something we got to get into with that, which is multi SKU attribution and measuring that versus having a single SKU and how you can think about all of these different plays. We’ll call them the book of plays. But if people converted and they thought they were buying something that had a shelf life of like, it was an infinite shelf life, it’s a piece of hardware that assume it’s going to work in perpetuity versus something that has a shelf life of a month, they might go, man, I converted and then they’re unhappy with the purchase and that leads to all of this negative sentiment around the product.

Ben Grynol (17:35):

So it’s the importance of, you can get people to do the thing. It’s not that hard once you figure out the creative and the conversion mechanisms, like what all those levers are in the right channels, you can get people to do the thing for a certain price if you’re willing to that price. But it’s all about what you said JM, which is around the idea of trust and retention and like what’s the LTV of that customer that you’re trying so hard to convert. So let’s go into this idea of having a company that has a single threaded product company versus multi SKU company, and then getting into some of these thoughts around it. So we’ll use the idea of driving awareness through things like a Super Bowl ad, keep riffing on that example.

Ben Grynol (18:20):

And let’s use Nest as the single threaded company. Assume this is like early Nest days, giving a nod to Chris Jones, but it’s only the thermostat. So when driving awareness and measuring attribution for conversions, you’re focused on a single SKU. So they’re tied together, like all the work that’s done to drive awareness and create trust and educate people say, here’s our brand, here’s what we do, here’s the product, the attribution touchpoint is still connected to the first touchpoint where somebody might have learned of the company. They end up converting, you’re working symbiotically between these like multiple levers that you’re pulling. But then you get into the gap. Or JM’s bodega, let’s use JM’s bodega. And it’s like a 5,000 SKU store because you sell everything from almond milk to you name it, bricks of cheese, everything. And you’re selling them online.

Ben Grynol (19:13):

Let’s do that. But you end up running placements like you’re running, paid social and you’re doing it for bricks of cheese. Well, you can measure the conversion touchpoint there, but then you’re talking about basket of goods. People are adding on other things because they came in through a single source. And so that’s where measuring attribution can be really hard and this slippery slope where data is very important, but it’s taking the step back of back to the Super Bowl ad, which I know this is like a weaving story because we’re weaving in all these different JM’s widgets versus bodega, but it’s the idea of, you have to look at this whole picture of what you’re trying to do when you’re talking about multiple SKUs, because it becomes very difficult to start to try to track for us the idea of what if somebody came in and they only, hypothetically this was possible, they only converted on blood work, but they learned about us through some tier one media placement?

Ben Grynol (20:06):

We won’t actually know that’s the case. So it’s where to play, given the SKU mix and then what you’re trying to achieve with it? So it’s just one of these things where give it… There’s no… I guess the takeaway of this little rant is that there isn’t one playbook. You can’t just run one playbook. You have to look at what are you trying to achieve as a brand? What is your value prop, your product? Are you carving out a new category? Do you have multi SKU arrangements? They’re all of these considerations and then what’s your end goal that you’re trying to get people to do? Is it a subscription business or a transactional conversion for a t-shirt? Very different businesses and very different considerations around getting that conversion and finding attribution for it.

Paul Barszcz (20:46):

I’ll go for another example. If we look at Walmart, for example, whenever you walk into the store, you have the items on sale, something that grabs the customer as they come in. And it’s sometimes the product that people have, the intention of buying when they come in but once they come in, they start to see other items on sale and with better margins. And I think that’s the key to Walmart’s success is they bring you in for one low cost item, but then they have so many other items out there in the store that you’re also interested in purchasing.

Paul Barszcz (21:19):

So I’m not sure about Walmart’s marketing, but I assume you wouldn’t see an advertising for one particular item, but more for the brand itself where they want people to come in to know that everything’s on sale or everything’s low cost and that’s their play. So I think that’s what some larger brands or the brands with many SKUs are doing. For example, we would do the same thing when I worked in a furniture business. We would have accessories with almost no margins. We would make no profit on selling them, but those items were at low cost and would bring people to the store and kind of have people know about the brand, learn about us, and eventually when they’re ready to make larger purchases, whether it’s a so far or bed, they would come back and do that.

Ben Grynol (22:13):

You’re loss leaders.

Josh Mohrer (22:15):

Yeah.

Ben Grynol (22:16):

Let’s dive into omnichannel. So this is something you hear often, it tends to surface on tech Twitter, or in DTC Twitter in the world of we’ll call it the world of marketing when people have these JM’s widget stores, they’re online, some of them are driving a few million bucks of revenue and they say, great, I found the channel. I’ve got it. And they start dumping all their money into one channel. And that’s okay for conversions, but what happens when you’ve got a single channel approach for capturing or for harvesting demand versus being omnichannel for other conversions or playing the full funnel game when there’s an outage like Facebook goes out or when the cost of acquiring those customers goes up significantly, all of a sudden people say, oh my gosh, I don’t know what to do. I lost so much revenue. I don’t remember when that Facebook outage was, but there was one relatively recently. I think it was the fall.

Ben Grynol (23:20):

And it might have been like a day or two days. And Facebook, this is Meta outage, we’re calling it because it was Instagram, Facebook, everything. And everyone who is running that playbook was saying, I don’t know what to do. I’ve lost all my demand. And so that gets into this idea of why omnichannel approaches are so important. And having this idea of almost we’ll a conversion portfolio. It’s almost like an investment portfolio where you want to have diversification to make sure that you are hedging against all these different channels, knowing that there’s also different traffic coming in through different sources and starting to think through how to play that game, as opposed to just going deep on a single channel.

Paul (24:05):

Yeah. And I think having that outage was actually a blessing for some brands because not many people want to run a test where they can turn off Facebook and see what the effect is, but given that it did turn off unintentionally and if they were mindful about it, they would’ve been able to actually measure the impact of their other advertising efforts and see if they’re actually getting any conversions through them.

Paul Barszcz (24:34):

One thing that I want to mention also is measuring incrementality is quite important as well in terms of attribution. A lot of businesses don’t do it properly or do it at all. So what’s measuring incrementality is essentially in the absence of this lever, how much would we lose? And so you’re actually answering questions. If I were to remove a certain channel, if I were to remove, for example, billboards from our marketing mix, how much business will we lose? And I think that’s one way, for example, to measure incrementality.

Ben Grynol (25:11):

Absolutely. Why performance marketing in general can be a slippery slope and Freakonomics crowds, Stephen Dubner and people that are doing deeper research into a lot of the nerdy academic aspects of performance marketing, analyze companies like I think Uber. I think it was around two, you’d have a better lens, JM, but it was around like 200 million bucks or a hundred. Call it one 150 to 200 million bucks, like a significant spend per year that it was just, well, we’re going to turn it off and see what happens. And it was like, oh, we found no difference.

Paul Barszcz (25:47):

I think they turned off 120 million. They turned off 120 million of their digital ad spend and it didn’t affect their app downloads at all.

Ben Grynol (25:56):

We should link this in the show notes. It’s this is in our earned, owned, and paid media memo and it’s a public document. And so it’s got the examples of a few companies. Big brands stopped spending on digital. It was sorry, it was Proctor and Gamble that turned off 200 mill, but there’s even a journal episode about this lens on Facebook and when the cost of acquiring customers went up so much for some brands that they stopped spending and they didn’t see a big difference. But yeah, Uber, it was that same thing where the engine was turned off and people just didn’t see a big difference because there’s so much to building a brand. There’s so much that comes down to running that Super Bowl ad, air quotes, because you’re building that trust and awareness and all these things.

Ben Grynol (26:43):

And at the end of the day, let’s just call the spade the spade, JM’s widget has to be an amazing product for people to love it. If people convert and the product isn’t great, then it’s all for naught. It doesn’t matter how great your conversion levers are and your attribution model. The foundation is and now we’re just pontificating a bit, but like build a great product that people love and then all the other aspects of what you’re doing become a lot easier so that when you can have the multiple touch points come with attribution, all the preliminary ones that being word of mouth, that being the initial awareness played through some, whatever, some billboard, some YouTube video someone saw that had nothing to do with the last touchpoint of conversion, but helped to build that trust. It’s a really hard game and so it’s so important to step back and realize that you have to really take this holistic view in building a product, educating people about it, and then getting them to actually convert is one mechanism of this whole game.

Ben Grynol (27:47):

And it just takes a really long time. What we’re seeing now, all of the conversions that we’re seeing now are an effort of two years of work of Casey and Josh appearing on podcasts and building this, especially because we’re creating a category. We’re part of creating a new category. It’s not something that people know. It’s like we started Rivian, a car brand, and people like, well, that’s a truck. I know what a truck does. Gets you A to B, it just happens to be electric and there it goes. It’s a lot harder when you’re asking people to do something that they have no idea what to do, but they want to learn more about.

Paul Barszcz (28:27):

Going back quickly to the earlier mention of Uber spending millions of dollars in ads and not getting the results that they expected. I think there’s a lot of brands that are doing this. For example, you brought up Proctor and Gamble and there was also Chase that cut their programmatic reach by 99% and saw the same results. So what they said is that they had ads on 400,000 websites and they dropped it down to 5,000 websites and they saw the exact same results. So what that means is we’re dealing with, I think this is every brand that’s probably dealing with this, is ad fraud. And there’s two types of ad fraud, or probably even more. One of them would be impression fraud and then the second one would be click fraud and a third one would be app install fraud. So it depends on the type of conversion metric that you’re looking at.

Paul Barszcz(29:16):

And there are different ways to do this. I don’t know them because I’ve never done this before, but from what Dr. Augustine Fu mentions on LinkedIn is that these people, they set up fake websites. And what I mean by fake is just, some sort of landing page and they load it up with several different ads on it. And these ads can come from different exchanges, such as Facebook, Google, and other third party ones. And what they do is they have these pages reload or have bots click on these ads automatically, or have some sort of method for downloading apps. And every time this happens, we as a brand pay for this conversion, whether it’s a penny or a dollar or even $5. And at the end of the day, we’re not seeing any performance out of this.

Paul Barszcz (30:09):

And this is quite a big problem. A lot of publishers or a lot of ad exchanges claim that their out fraud is not that high. But according to Dr. Augustine, he mentions that it could be anywhere from 10% to 99% of fraud happening. There’s a lot of research he’s done in the past around this and other people as well, and it’s quite the problem. So where am I going around with this is if we’re looking at our attribution, we got to make sure that we’re actually measuring the performance of our channels and not only looking at one or two, several metrics where it’s did this person buy or whatever, but also try to look at our impressions and see is the number of impressions that we’re getting, is it really high and is our conversion rate really low?

Paul Barszcz (31:00):

And if it’s really high, you want to ask yourself, are you paying too much for these impressions? Is your CPM price too high or even is it too low such that let’s say you’re paying a dollar per CPM, is that too low and you’re just getting jump traffic? So you just got to, I guess, on a regular basis kind of prune your ad spend, and also make sure you know where your traffic is coming from and set up some filters or ways to kind of clean this up.

Josh Mohrer (31:31):

I got a good story from the Uber days that I was just reminded of. We had a referral program there in a way similar to what we have at Levels, except there was a reward. So if you invite someone to Uber, they get a few bucks and you get a few bucks when they take their first ride. And so we had these people who would buy ads against their referral code. So they’d buy like a Google ad, like try Uber, get $10 off your first ride with the promo code, JM’s bodega, and people would go and do that and then we would owe them tens of thousands of dollars in referral credit. And there were sort of a question of is this good for us? Is this bad for us? It kind of seems bad. Someone is grabbing the last touch attribution to get the credit, but it’s also kind of interesting, they’re taking a risk on it.

Josh Mohrer (32:22):

I was kind of reminded of that because this idea of view based attribution, I haven’t bought a digital ad in about a decade, so I don’t know if this is still a thing, but my recollection is that Facebook and the like can tell if someone even just had your ad on the screen and then you saw it but didn’t take an action and then later you convert through some other means, Facebook or the ad seller or whomever would say, hey, like our ad helped do that. So maybe we should talk a little bit about view based attribution and there’s a little bit of shade in there.

Paul Barszcz (32:57):

Yeah. And then I think that’s the problem as well of a lot of brands. I would say brands that are not super well advanced in the way they do their ad spending. You can spend money on Facebook, Google, Pinterest, Snapchat, and everywhere and you let’s say your attributional windows seven days. If someone makes a purchase within those seven days and they saw each one of those ads, let’s say you’re remarketing to everybody on every single one of those channels. Each one of these platforms will try to claim that purchase. And in each of these platforms, you’re going to see one sale. And then when you aggregate all of those sales together, you’re going to see like, oh, look, I had six sales, and because Facebook has one, Google has one everyone.

Paul Barszcz (33:43):

But then when you look in Shopify or whatever e-commerce platform you’re using, you only see one sale out of that. And so you got to ask yourself like, which channel actually influenced the person to make that purchase. Is remarketing actually worth it or did the person already have an intention of buying it and they just forgot. And all they needed was for example, an abandoned cart email? Something like that.

Josh Mohrer (34:08):

Yeah. Well I think abandoned cart is a really good example. We don’t do that right now. We’ve experimented, and we probably will, again. That is like the last touchiest touch. Just you were almost there and we’d give you a little push. And you asked the question, which one should get the credit? If you have all these different things that say, we helped because we did a view based attribution cookie and we saw that they went through after or we do the referral program or we do a reminder email. The answer is kind of all of them. And I think you can do some work to figure out which are the most responsible. But I think it’s hard to know even subconsciously if someone sees an ad on their screen, and doesn’t click.

Josh Mohrer (34:47):

It’s just one more thing like, oh yeah, I should do that. Which makes it hard to measure, to be honest. It’s hard. And in some ways, the only way to really know is to have everything you’re doing and then stop one of the things and see, I mean, it’s not the only way to figure out, but it’s one way to figure it out is to stop one of the things and see what happens. There’s probably a bit of a half life on out. Like when Uber stopped doing whatever it was doing for ads and oh look, everything continued. It kind of did, but also the value of those ads were over time. It wasn’t just for that moment. So all the advertisements and all the brand stuff up until then is what led it continue to survive without the ads. So it’s a little bit disingenuous to say, oh, we pulled the ads and everything stayed the same. We didn’t need them after all. No, you did maybe at the time, but maybe not anymore.

Ben Grynol (35:35):

Exactly. And that’s for this idea of performance when it’s related to conversions that you’re trying to measure through an attribution model. It’s probably more along the lines of the game that a lot of prominent DTC companies are playing now. We’ll use the early two thousands to late nineties, early two thousands into today and what that looks like. And it’s the idea of ignoring a lot of the last touch attribution marketing initiatives, because you have to plan for them, you have to design for them and say we’re hitting this channel because we’re trying to hit this demo. We’re trying to target them in this way and get them to do this behavior. What happened in the late nineties, early two thousands was a lot of these big brands that were consumer brands were doing the Oprah effect. If they could get Oprah aligned with their product and Oprah would do the under the seat giveaway, that was kind of like the market maker for them.

Ben Grynol (36:28):

That was the holy smokes, look what happened to our company. Well, you can’t really measure that. You just know that generally things are up and to the right for your business because of this thing and it happens over time. Now, fast forward to today, the game that a lot of these DTC and consumer brand companies are playing is the Oprah effect of, it’s mostly the podcast world, but it’s like the Joe Rogan effect, the Tim Ferris effect. It’s if you can get aligned with these companies, let’s throw out the door, well, it’s actually a real scenario where we want to know that there is some attribution associated with the traffic that’s coming from any source that any of the prominent thought leaders that we are aligned with but the game gets a lot different when we’re not offering incentives for people to convert.

Ben Grynol (37:16):

Once we are post liftoff, that’s not the game we’re going to play. We’re not playing the “and get your free,” right? We’re playing the hey, this person believes in what we’re doing and they have enough trust in us as a company that they’re saying, try this out. That’s a very different game. And it’s still it’s saying, we’re not going to go deep down the dumping money into Facebook to try to trick people into converting and that’s maybe a thing of right now, but it’s just the time you invest in trying to create these attribution models and understand conversion is totally different because we invest all this time in trying to create the partnerships and still have some measurement. But we’re not trying to like dissect the data at the most granular level because it’s back to everything that we said from the beginning where it’s like, there’s so many touch points associated with this where it’s like, generally people agree.

Ben Grynol (38:07):

If you can get the Oprah effect, like if you can get the Tim Ferris effect and that feels like a flywheel that’s actually working, it’s like, there you go. You’re hedging against ad fraud where they’re basically forcing your ad budget to be spent on all this garbage traffic that’s not actually leading to any net benefit for you. So anyway, the idea of spending in the right places on the right things, the Super Bowl ad, if that’s the right thing for JM’s widget co or the, whatever it is, the billboard and the handshake outside of JM’s bodega to get people to convert on the brick of cheese, there you go. These are all things that make a difference in the long run. Spend your time and your capital in the right places and know why you’re spending them there and that’s somewhat the lens on how to think about these things.

Paul Barszcz (38:54):

I think there’s different levels of doing attribution. You can do very basic one, starting with surveys. Just basically asking people after they make a purchase, where did you hear about us? And that’s like the most basic way to do it. And then some platforms out there have the last touch attribution model, and I think sometimes you could pull the first touch and other elements in between. And then afterwards you have the pixel tracking and URL parameters and all that stuff and discount code tracking.

Paul Barszcz (39:23):

So I think having all of those basics in place is important, but then I think getting too hung up on attribution is also a waste of time. You got to understand if your channels are performing or not, but I think kind of move on and keep growing your business and kind of pull on different growth levers rather than just focus on measuring all the time. A lot of people can argue against this, but that’s kind of my philosophy on things is keep an eye on your analytics and your attribution, but don’t dwell on it on a daily basis and just keep grinding and growing your business.

Ben Grynol (39:58):

There was a really interesting thing, JM had the closest lens in the company on everything around David Sinclair. So if anyone didn’t hear David Sinclair’s podcast, Lifespan came up with this podcast. We sponsored the entire season and it was one of those things where you can have a heuristic of, I think based on some analog, I think it’s going to perform in this way and it doesn’t matter how much you try to forecast any black swan there’s a reason black swans are called black swans. And David Sinclair was a bit of a black swan for us. So when it started, when we started running these placements, we were getting a lot of traffic and it was somewhat surprising. So I think JM, initially you were looking at that basically every day for the first however many weeks.

Ben Grynol (40:46):

And then as it’s tapered off, you still got a lens on it. But why don’t we talk about this is the bull case for why having some attribution matters because there’s a lot of value created in those leads because there is some purchase intent or some interest. We’ll call it interest. There’s some interest enough in the company that people decided to take an action. So why don’t we riff on this idea of, we sort of made this like bull case for, hey, attribution matters, but it kind of doesn’t sometimes. And then there’s also the other side of, we framed it as the Oprah effect. It’s, and I’m being hyperbolic about it, but we framed it as yeah, attribution matters, but they’re, it’s very nuanced. And then now let’s go into the David Sinclair of yeah, man attribution matters. Here’s the bull case of like, why you really should understand your traffic.

Josh Mohrer (41:36):

Well, here was the interesting thing about the Sinclair ad set. And if you’re not aware and you’re listening, we were a sponsor of David Sinclair’s season one of his podcast about life extension, essentially. And it was great and a ton of people listened and it drove a lot of volume for us. What was interesting at the time is that we generate demand in a lot of ways. Our blog is getting bigger every month. Something like 250,000 visitors a month now. That’s mostly organic. Someone is searching for something on Google and get a link to us and they come and read and they learn about Levels. And we do a bunch of other things as well, but you can’t just come to our website and join because we’re in this beta period. But when we do these partnerships with a Sinclair, that’s sort of an exception. We let folks in through that.

Josh Mohrer (42:27):

So I think we generate demand in all these really interesting and organic ways, and then we went and did the ad campaign and it did extremely well because I think one, he has a huge listenership that was interested in his opinion and when he endorsed us that made them want to come and do it.s Presumably that’s the bulk of the people. But there’s also, we also saw that a lot of the buyers through the Sinclair ad were actually already on the wait list. In fact, that tends to be the case with a lot of our purchasers. Once they find a way in, they oftentimes were already on the wait list.

Josh Mohrer (43:00):

So in some ways we gave Sinclair that last touch attribution on a bunch of memberships and we don’t know. I think some of those folks who were on the wait list probably would’ve converted with just an invitation. Maybe they needed one more kind of thing before they wanted to do it and hearing Sinclair talk about it, made them interested. But it’ll be interesting to see kind how these things go when you can just visit our website and sign up. That’ll be interesting. Our data on performance for ads at this point is biased a bit because you can’t get in any other way. They are both letting folks know and are kind of the sole entry way into our system, into our membership right now. So there’s an interesting one there.

Ben Grynol (43:45):

Yeah, it’s the trust factor, because that’s what you said is, what was it, 35%? 35% of the… It was somewhere around there.

Josh Mohrer (43:53):

Yeah. It’s something like that. The number of people when they joined were already on the wait list and there was another thing people who joined often almost bought before and then maybe they abandoned cart for some reason. We just see, it’s never as clean as I turn up, I get an opportunity, I buy, I go off into the sunset. There’s always a path there that we’re going to get smarter about figuring out and understanding, I think.

Ben Grynol (44:15):

One of the interesting behaviors was, it’s something that we see very often, this is something that will likely exist even into the future post liftoff is the idea of there is a direct correlation between placement, we’ll call it placement date and given like podcasts have a window that people listen, let’s not even get into all of that, but there’s a direct correlation between placement date and conversions and email signups. The spikes are there. Basically you can directly overlay them. One is a little bit bigger than the other, naturally. But what that means is this is where you have to start to glean your own insights from these things. Did people sign up via email because they didn’t know there was a conversion link? Did people sign up email because they had a longer lens on purchase intent?

Ben Grynol (45:06):

Did people sign up via email because they actually just want our blog content on a regular cadence, that being like our newsletter? They’re all of these things. But the main takeaway or the main insight to anchor on is, hey, somebody cares enough about this thing we’re doing to have given us an email, and that is a lead of, with some sense of value to it. And then it takes a long time to continue building that trust. So it’s like they’re already in the funnel, but that’s where it gets really dangerous to be like, oh, we didn’t see a ton of conversions, hypothetically. We didn’t see a ton of conversions from, Paul’s got a podcast. We saw a ton of email sign ups, but we didn’t see a ton of conversions from the link. And then you get into the idea of, well, was that a bad placement?

Ben Grynol (45:52):

Because it didn’t convert. And it’s like, well maybe it actually did the right thing for us, it helped to build this foundation that we’re continuing to build. It gets really slippery to be on the slope of just like, well the last touch didn’t work on this thing so conclusion, therefore it didn’t work. And that’s why we debate this stuff internally so much when we start to, especially with some of the takeaways or learnings from experiments, which JM’s getting deeper on as far as bringing rigor across the company for these things so that we can really debate them. Because it’s important for us to not just subscribe to one conclusion, like Ben concludes that this thing works or doesn’t work. It’s like no let’s challenge these assumptions so that we can start to dissect it. Because when we do, we might come up with a totally different takeaway or conclusion.

Ben Grynol (46:42):

And we find like down the road, if we didn’t challenge each other’s assumptions, it’s like, oh that thing that we didn’t think was good actually was amazing. That being like hypothetically Sinclair didn’t lead to a ton of conversions, but the traffic was really valuable or the email signups we got were really valuable long term leads that as a cohort analysis, if we measured email signups, we’re like man, like everyone who came through that traffic source has an LTV of $10 billion, just something absurd. But we’re like holy smokes, that was a really valuable thing. So thinking through it with a really wide lens is, and being able to have enough people with oversight and a different lens as far as strategy and insight and input on these things is important. And I think that’s a neat thing about the stage that we’re at is we’ve got so many smart people with different views on these things that we come up with good conclusions, I think to put forth these partnership and these marketing initiatives that we’re undertaking.

Paul Barszcz (47:42):

Yeah. Having a look at the customer journey in general is important and understanding what touchpoints have the impact on someone actually converting and realizing they need our product. So you can have one partner do a podcast and the person is convinced right away because of the trust level they have with them. And you could have one touch point at a given time, but I think some people they need convincing or more education kind of understanding what we do and why they need our product. So I think being able to surface the idea and just the concept in general of metabolic health, educating people on that in itself is not easy for different type of people. Some people might not even know what it means, so we have to educate them from the basic level.

Paul Barszcz (48:37):

And some people might already have an understanding of it based on the readings and their knowledge in general about it. So I think just getting a grasp of how long does it take for a person to learn about the concept and then converting is important as well. And some people, it could take a few days or some people, it could take a few months. I can give you an example with Athletic Greens. I first heard about it through Tim Ferris many years ago and I’ve never purchased it. And only recently after hearing other podcasters promote that product and everybody at Levels drinking it, that I actually went in and bought it. And after buying my subscription, I used the promo code that I just found on Google. I just used the random promo code to get whatever bonus I can get from it.

Paul Barszcz (49:26):

So if Athletic Greens looks at my touchpoints, they’ll only see one point where it’s Paul visits our website and then he comes back through a link on Google, some coupon site linking back. And so their assumption is like, oh, okay, this guy came out of nowhere and found this on Google and made the purchase. But in reality it took me several years from the first touchpoint of Tim Ferris to more recently several months here at Levels, seeing you guys drink all of it, drink it quite often. So those are touch points that Athletic Greens doesn’t know about and they never really even ask me at checkout whether where did I first hear about the product. So that’s why attribution is not easy and it’s more of a social science thing than a hard science.