#FakeStats, #FakeNews And That Yelp Survey
Those of you who read TV[R]EV on a regular basis know that shoddy/specious research is one of our main bugaboos. Clients pay us to provide them with informed opinions about the future of television and when we try and find research to back up those opinions, so much of what we find is poorly done: either the sample size is tiny (e.g., "we interviewed 35 people and made sweeping generalizations about the media habits of all 300 million Americans as a result") or incorrect assumptions (people switching to one of the new virtual MVPDs like DirecTV Now, Charter Stream, Hulu Live TV and Sling are "cutting the cord".)So we were not surprised to see that a story flagged on LinkedIn about how a study of Yelp users shows that "mom and pop" style restaurants are more popular than chain restaurants and that the gap is growing.The results were based on the number of reviews that Yelp users left. And so we're reading this thinking "this is no doubt true, but ...."
- Yelp is more or less designed to help people find small mom and pop style restaurants. That's sort of its whole purpose. So it should not be surprising that those types of restaurants get more reviews than chain restaurants. If anything, we'd have expected the gap to be wider.
- McDonalds and Olive Garden and the like all have hefty ad budgets to let diners know what's on the menu and what the restaurant looks like before they get there. That disincentivizes Yelpers from leaving reviews for one of those places unless their actual experience was very different than the one promised by the ads.
- Heavy Yelp users are the sort of people who pride themselves on knowing about all the new trendy restaurants and/or the really good off-the-beaten track ones. They are not the people who pride themselves on being intimately acquainted with the menu at Ruby Tuesdays. So again, no surprise that they're not leaving reviews for Burger King.
We're flagging this story to show why it's so important to take a critical eye to any sort of research, even when it comes from a reputable source like Yelp and even when it's most likely true: there's often a reason why something is true and it's important to understand whether that reason has applications beyond that single instance or if it's an outlier, as it is here. We'd like to think that our dedication to doing that is why clients trust us.Interesting sidebar: In doing a bit more research, I found this piece from the Washington Post which had a surprising (to me) fact in it: "There are wide variations by city: New York’s restaurants are largely independently owned, with only about 35 percent of its fast-food restaurants part of a chain, while Phoenix is chain-dominated, with 80 percent of its fast-food restaurants operating as part of a chain." That's the sort of stat that speaks to bigger differences and experiences across local markets and may be relatable to media consumption habits in those markets. Or not, but it's definitely worth wondering about.
