Transcript

Paul Ford You know why leaders love data? Because it’s not their decision. Because it’s just about the data. What are you going to do?

Reed Whitmont It makes it easier for everyone involved.

PF It’s so good!

[music ramps up, plays alone, fades out]

PF Richard! 

Rich Ziade Yes, Paul.

PF How are you doing? 

RZ I’m doing well, man. I just, I’m in sweat pants. I’ve got a lot of sweat pants. So I’m in  sweat pants. 

PF I wasn’t looking for that. But I got it. 

RZ Here we are.

PF We’re work with it. So listen, do you ever put like a website out in the world or put a product out like maybe in the app store? Ever had that? Ever done that?

RZ I have, to varying levels of success, to be frank.

PF Extremely varying. [Paul & Rich laugh] What a world! So after it launches, okay? Now what? How do you know, okay, we got it. We got it figured it out. We figured it out. We’re gonna get some users. What now, man? How do you know what happens next?

RZ You know, depending on when that I’ve been in the business for 25 years. And for a long time, it was one line of JavaScript for Google Analytics. That’s all you had.

PF You just dropped that in there, that’s right.

RZ It was crude. It didn’t speak to success.

PF You could grow your own. 

RZ See what’s coming to the site, as if it’s, you know, a blog. So it wasn’t great. 

PF That’s the thing, it used to just be real gross anatomy. Like how many people looked at pages? That was what mattered in the world.

RZ Hits. 

PF So look, you and I were talking, we talk about analytics, we talk about product all the time. And it turns out we have some very, very analytics oriented people at Postlight.

RZ Interesting, like who?

PF Yes. Wacky coincidence. Reed Whitmont is on the podcast with us today. He is in the product management group at Postlight. Reed, welcome. 

RZ Welcome, Reed.

RW Thanks, guys. Good to be here. 

PF You know what’s great? Reed has one of the voices like I do, where people are just like, people when they read me, they love me to talk like this, because I’m a big nerd. But then, you know, I’m like, well, you know, let’s talk about product read. So this has now become a cool jazz podcast.

RW You got to come in like that. You just have to.

PF There we go, yeah, this is exciting. Let’s talk about the product. [Reed laughs] Richard, your voice, you’re like an animal compared to Reed.

RZ I am an animal. Can I share a feeling that I think many product managers feel? A lot of product management is the bobbing and weaving and ducking and jumping to get to the line. And then when you get to the line, you expect this moment of euphoria. And it is euphoric in that moment, but you know what’s really depressing, Paul, and Reed, you know what’s really depressing? Day three. [Rich laughs]

PF Oh, no, I’ve actually learned, it’s hard. Because you actually have to coach the clients, you have to tell them like, hey, post launch depression is a thing. But you know what, before we go in there, Reed tell us about what you do at Postlight, tell us about your job. Tell us what you do all day. And then we can before we drill all the way into analytics.

RW Sure thing. Yeah, I’m on the product management team. I’m a Senior Product Manager here at Postlight. So I do a whole wide variety of things for those listening, and really just help product teams guide strategy and get to a place where ultimately things, beautiful things go live. That’s a hyper reduction.

PF That’s good. That’s perfect. Help beautiful things go live. Why analytics? Turns out that you know a lot about analytics. Where does that come in?

RW Yeah. So honestly, in a previous life, I was much more on the marketing side of things. And I really started out in what would be called, but I don’t like calling growth hacking, which has some it has some overtones. 

PF Whenever I hear that I just think of like a dermatologist, just hacking growth.

RW That’s better, honestly. I like that more. But what I do really like about that background is it’s very, very data heavy. And one of the things that I’ve really noticed as I’ve transitioned more on to the product side, is the extent to which data is there. I mean, it’s in the room. And there’s you know, there’s there’s very few people in tech who you have to convince that being data driven is a good thing, everyone knows, but the extent to which people are actually data literate, and the extent to which I’ve worked on more accounts than I can count. And yet, I think there’s maybe one or two that have had good analytic setups. And so I’ve just, it’s a language that’s always spoken to me as someone who’s not a data scientist, but it’s a place where I’ve always seen, it feels like somewhere that people are often very confused and overwhelmed. And to me, it’s always been just fun, and I geek out on it. So it’s a place that I’ve naturally just dived into more and more over the years. 

PF Define good. What’s a good analytics setup?

RW It’s a great question, because I think they’re both isn’t isn’t a clear answer here. There is because a good analytics setup is one which tells you what you need to know. And what you need to know is, is every visitor tracked? Are you able to see how every visitor is moving through the pipeline through your funnel, where they’re coming from? Are you able to segment them and look at them in cohorts and understand how different user behaviors signal different outcomes, things like that. But at the same time, one of the biggest challenges I tend to see in analytics setups is people over complicating and being unclean with the way they set up their data. So that the things coming back are just, there’s a very common but very accurate expression, garbage in, garbage out. And essentially, if you to put that another way, it’s very much a chicken in the egg situation, in some ways for a lot of people, which is to say, you need to know in some ways, what you want to learn when you’re building your setup. But without that good setup, you won’t get that information. So you have to really build these things holistically.

RZ What’s funny about features, when it comes to these platforms, these analytics platforms, is I call it the 747 dashboard. I mean, you drop them in, and then all of a sudden, it’s a flood of not just data, not just raw data, but it’s just a flood of like switches and filters. Some of them let you daisy chain the filters. So you’re filtering this and then you’ll filter it again. And then once you got it to a good place, you can save it, right? Like so a lot of the hard work, I feel like with a lot, that’s what I mean, there’s an industry, let’s point this out. Like this isn’t just something that’s a corner of Product Management. There are experts. Postlight has hired experts in the past, to help us decipher the data that’s coming in. Sounds broken to me, if I’m a product leader or a product marketer, Reed, And I want to get to a place where I can just peer in and make sense of what’s going right and what’s not. How do I navigate that? I feel like dashboards are really the counter argument for all that complexity. It’s like, well, executives can’t stare at walls and walls of stuff, right? We got to give them a dashboard, which, okay, fine. Take my hand here and walk me through. How do I not drown in this?

RW So I’d say if I were coming into a situation with someone else’s analytics setup that I didn’t build myself, which is generally the case, and quite frankly, with most long term clients I’ve ever worked with, I’ve ended up rebuilding the analytics setup. So that’s that’s a side thing. But coming in and trying to find good data, I first need to validate, does this analytics setup actually work? Like, am I going to get data from this that I can trust? So I’m going to check a few things. I’m going to check, is the implementation correct? Is this showing up on every page is every visitor as far as I can tell tracked and there’s numerous ways to check that, but generally speaking, is this catching everybody, and is it catching everybody correctly? So for example, one of the simplest errors that I see a lot is the tag, which is the actual bit of code that lives on a page that sends data to an analytics platform, firing or loading at the wrong time and actually creating a user session that looks like something other than it is. So for example, there’s a lot of analytics setups I’ve seen where you’ll have a 0% bounce rate, which is the amount of people who come to a page and then immediately leave. And actually, that’s not true at all. It’s just the tag is firing in a bad way. And so the data is bad. So right off the bat, if I see something like that I know, okay, there are parts of this data I can’t trust, I want to check every single stage of my funnel and make sure that the data I’m seeing makes sense from what I understand. But also I want to run some tests on that to make sure, is this data accurate?

RZ How do you run those tests? Is there a tool you like to use? Or is it just like point and click and go check? Like, how are you validating?

RW So it depends on the scale and on the tool I’m using, but let’s use Google Analytics as an example. Because it’s it’s kind of the industry standard here. And honestly, I think the best. So with Google Analytics, I would do two things. Say I was auditing the Postlight analytics setup, I would do a few things, I’d use a kind of test view. So I’d set up a view that only sees my IP address. And I’d run through the funnel and make sure that my session is being tracked at every stage along the way. I would also submit some fake form submissions and make sure that those are coming through correctly. And then I’d also compare because very, very, very, very few websites and products nowadays exist in isolation. So we have form submits, those forms summits are going somewhere else so I checked to make sure that the number of forms submitted the timing on those form submits on our back end is the same as what’s coming up in analytics. And there’s a lot of ways just to kind of sense check this. And I’m, I’m not even looking for exact, exact, exact same, though, if I don’t see that I’m going to be concerned, but just making sure hey, is this data showing me what it should be? And are my traffic sources where people are coming to the website from? Are those all being properly labeled? There’s a lot of places where people tend to make very simple mistakes that have very lasting consequences. Like when they’re creating, for example, paid advertisements, you have to create something called a UTM, which is an urgent tracking–

PF It came before Google Analytics. Wait, hold on me pause began sounds like Google Analytics is kind of the benchmark. Like when you’re saying like, you’re gonna need to make UTM like that. That is the industry like, is that still true? Like if I’m thinking analytics, am I just thinking GA are there other platforms that I’m oriented around?

RW So there’s tons and tons of platforms in this space, GA is my favorite for a number of reasons. But I’ve worked with, oh, gosh, 10, 15, 20, different analytics platforms and all, all serve different use cases, GA is my favorite for web properties. But it’s probably my least favorite for mobile apps. And when we’re talking about things like UTMs, all of these things in terms of the way we’re implementing tags in terms of the way we’re tracking things, these are relevant, regardless of what thing you’re using, for the most part, most of them function pretty similarly, it’s just different ways of data visualizing. However, there are a number, segment is a good example that do function in some fundamentally different ways in terms of how they capture the data. But I always advise clients to steer towards Google Analytics, because it’s the most robust in terms of the amount of data it gives you. And also one of the and there are people who are going to hear this who are going to very strongly disagree with me, but I think one of the simplest to implement, because it’s been around for so long, and the the documentation around it is really just so robust. And it just works. It just works very well.

PF So a lot of our listeners are PMS, but a lot of them are going to be, you know, coming at this from various different angles, talk about some of the things you can measure, right? Because it’s yes, you can pay this, IP addresses unique users, I think, like people understand that. But now I can instrument just about anything. So like, what are we measuring? Like, objectively? What are those things?

RW That’s a great question, because I think, honestly, one of the things I see most around analytics setups is people who implement or people who are responsible for reporting on metrics getting so excited about all of the data points they have, and also kind of getting overwhelmed and really just not knowing where the story is. And I always recommend taking a huge step back here and going back to basics. And what I mean by that is, at the end of the day, the most important thing is what is your funnel look like? And where along the funnel, do you have the largest opportunities for improvement? That’s the single most important thing? And, surprisingly, that’s a question that almost no business I’ve ever worked with, has really been able to answer. They’ve had stories about, oh, you know, we’ve got a conversion problem. Oh, you know, we’ve got a traffic problem. But then when you look at the actual data, it’s often a very different story.

RZ Let’s take 60 seconds and give people the one on one on funnel. The idea of a funnel, I think predates the web. I think funnel is a sales funnel, it’s a marketing and sales term.

PF Originally it’s a V-shaped sort of cone thing that you would pour liquid or solid into in order to make them.

RZ Exactly. But now, and this ties into the term user journey and all that. But what is a funnel?

RW Yeah, so a funnel, and that plastic or glass or metal implement in the kitchen is actually a great thing to think about here. Because the idea is at every stage of the user journey, so that’s from a user having no idea that your product exists to a user, being a dedicated long term user and actually referring other customers, it gets a lot narrower. So you’ve got, say, 100 people in the world, that’s the population of the world. And of those maybe 50 know about your company, because we’re just knocking marketing out of the park. And of those 50, 25 come to the websit, of those 25, five of them click on the call to action button and create an account or whatever our thing is, of those five in a year. One of them is still there. And maybe of that one, well, I should have picked a larger sample size, but the whole idea–

PF It’s going from a really big number to almost always a really, especially in things like like our work right where we we might talk to 1000s of people a year and 20 of them would become a client. It’s going from large groups and then as they go through the funnel, they have more and more of a relationship with you.

RW And the important thing here is when you’re thinking about that funnel, if you actually can map that out and look at, okay, my acquisition rate is this, my conversion rate is this. My retention rate is this. Every stage of the funnel, if I can put a number to that, what I can do is then I can look up, okay, what’s, what’s the industry benchmark for agencies converting on their websites? What is the all of this stuff exists online, and then I can see, oh, you know what, I really thought that I was converting badly. But actually, I’m converting relatively well as to be expected, I could get a little delta there. But actually, there’s a significantly larger opportunity. At this other place in the funnel, maybe it’s getting people to add this certain amount of information, or maybe it’s getting people to the page to begin with. And just that alone, it’s it’s such a simple thing. But the amount of businesses I’ve worked with, who have been able to just show me their entire funnel from end to end with that data is almost none.

Reed, I’m going to be our listeners for a second. Because here’s what they’re doing. I work at a company that has a big website, and they talk about analytics by geography. And they have some rough sense of personas, and the website has 200,000 pages. I like this guy, he seems to really know his stuff. But how am I going to convince like, we’re not really thinking funnel? Yeah, sure. We do some marketing and sure. But how am I going to convince, because this is a big job, I got an instrumental those pages, I gotta add an analytics to them, how am I going to convince someone that they should go and put that energy in when we’re not quite sure exactly what we’re going to get on the other side? It sounds like, because what you’re selling me is better decision making on the other side, as opposed to like just instant revenue, which is what everybody really wants, right? So like, make that case.

RW Yeah, that’s a great point. And honestly, I over index, perhaps on the simplest stuff, because that bedrock is often not there. But where it gets really exciting and where it gets into, I don’t want to say more immediate revenue impact, because funnel analysis can give instantaneous revenue impact if you shift your resources around. But where it gets really cool and potentially easier to sell is, once I have that funnel analysis, and I’m going to I’m going to go heavy on that funnel analysis. But once I have that, I can say, Okay, let me look at my users. Let me break this down into cohorts. Because especially in Google Analytics, I can see user demographics, not necessarily at a personal identifiable information level, although depending on the setup, sometimes you can, but tracking and data, personal data are two very different things. But I can see, okay, you know, what, here’s my funnel, I know that I’ve got a conversion problem. And that’s, that’s where I’m going to make the most money. So let me look at who is converting and who’s not. Hmm, I can see that users between the age of 25 and 35, who live in this part of the country who have these affinity categories, which means that their online search behavior, really lines up with certain products or services, oh, I can see that those people are significantly my best converters, whereas this group of people is really hitting a problem at this stage. And with that information, and of course, I’m oversimplifying this here, but with that level of granular information, I can build customer personas that let me know, okay, I’m having a beyond just knowing, okay, my customers are X, Y and Z, I can actually know, wow, this customer, so say, middle aged people from the middle of the country, I’m just using the word middle a lot, we have really high interest from them, they come to the page a lot, they have a lot, they look at a lot of pages, but they’re not converting. Well, that tells me very clearly that I’ve either got a product market fit problem here, or more likely, because they keep looking, I just have an education problem here. There’s some piece of information that they don’t have, that would make them feel comfortable with taking the next step. Because if they’re if they’re showing that level of engagement, chances are, they’re seeing some value here, they’ve got some interest, but there’s something missing. So if I look at the people that I’m serving, and how they’re interacting against my funnel, and again, that funnel is critical here, because that’s, that’s really how you connect it back to revenue, then I can say, if I better serve by increasing this one metric for this one user base, this is the impact it’s going to have on my bottom line because when you have that funnel analysis, I can literally say okay, if I increase this by .5%, that’s going to mean X amount of dollars at the bottom of the of the funnel.