747: Taking Metrics Too Far

00:00:02
Speaker 1: This is the Late Podcast with your host JJ zacheras Jake Zachers. What's up everyone, It's JJ zacharyesin in this episode seven hundred and forty seven of the Late Round Fantasy Football podcast sponsored by FanDuel. Thanks for tuning in. I know I usually published these early week episodes on Tuesdays, but this one's a little bit different. I've got some things that I want to say to the universe. First, the Late Round Prospect Guide is done. It's out there, it's finished. Anyone who pre ordered it, it's now in your inbox. Make sure to check your spam if you don't see it, and anyone who wants to get it can do so over on lateround dot com right now. The price, though, is now fourteen ninety nine. When you order, it'll immediately be delivered to your email inbox. Now, this episode is sort of tied to that Prospect Guide. The metrics are going to be talking about today are metrics that you'll read about in the guide, but that guide's also me. It's my brain, my process, the way I think about this stuff, and that definitely ties into what I'm going to be talking about on today's episode. Fantasy Football is a numbers driven game, and you hear me analyze everything with numbers two, I mean that entire prospect Guide is built through data and math and numbers. Because everything is so numbers driven, it can probably be overwhelming for some people, especially people who don't like math or who aren't good at math. Some people are good at writing, some people are good at art, some people are good at math. Everyone brings something different to the table. I try my best on this show to make things as consumable as possible. I've said it before and I'll say it again. I'm not really a data scientist. I'm far from the best data analyst out there. I just think that part of my competitive advantage comes in the form of communicating ideas. So I'm going to try to communicate one of those ideas for you today. Something that I've noticed after doing this for over a decade now, which is crazy to say, is how people seem to latch onto ideas and just run with them. Idea gets thrown out there, and if it's an exciting idea, people immediately want to try it out, which is reasonable. But when there aren't parameters being set around that idea, it's very very easy for people to then take things to an unintended extreme. A few years ago, I did some research and put together what I called the ambiguous RB one theory. The theory stem from the idea that in the middle rounds of your fantasy drafts, you should be targeting running Max from ambiguous backfields, but on top of that, you should be targeting the team RB one by ADP over the team RB two. So if there are two Philadelphia Eagles Running Max getting drafted in the middle rounds, the one who gets selected earlier is the better bet. That's what the data said. I published that study and I immediately regretted it. What I was trying to say was that we should be targeting ambiguous backfields first and foremost. But given the ADP source that I was using, it seemed like the fantasy community ends up getting those backfields right by ADP more times than not, and it wasn't full proof. And my mistake was that I was looking at things in hindsight because as summer months go on, ADPs fluctuate and change, So instead of the focus being on ambiguous backfields and ambiguous backfields only, which is a very very important concept, the focus shifted to which running back on this team is a higher adp which one is the RB one for the ambiguous RB one theory. And that was my fault, one hundred percent my fault. I presented it in the wrong way. I didn't think it all through. Even though that data was worthwhile and interesting, it wasn't something to latch onto the way people latched onto it, and people latched onto it because of the way that I presented it. I'm not immune to this. The way analysts present information is unbelievably important because when these metrics and ideas get introduced to the world, people will run with those metrics and ideas, and when it's not presented in a rational way, it can do a lot of harm to people's fantasy football processes. Another quick example was with early declare status. More and more analysts and fantasy managers have become aware over the last four or five years that there is signal to early declare status, that when a player leaves school early for the NFL, it's a sign of talent and those players generally perform better than players who stick around. But again, people have taken this idea too far. Just last season we saw Chris olave get pushed down in rookie drafts, and a lot of the reasoning for that was because he played that extra year in college. And I'm not saying early declare status doesn't matter. I'm not saying I have all the answers either. Early declare status is an input in my model. This is all about the degree in which something matters. Let me walk you through a more in depth example. Last week, Cameron from the Late Round Fantasy Football Patreon asked me a question in the site's discord. He was noticing tweets and Reddit posts about speed score or wait adjusted forty times and the correlation between speed score and running back success in fantasy football. That led to this question that he posted on Patreon just last week. It said, Hey, JJ, you talked frequently about wait adjusted forty time or speed score and how you look for players to hit a threshold of ninety. However, some analysts think that there are additional thresholds at one hundred and one hundred and six. Why do you use ninety as your bar instead of using multiple thresholds? Now? I was going to answer that question on last week's mailbag until I started answering it and I realized that I had a lot to say. So let's start by answering that question directly. Why is my model's speed score threshold ninety and not one hundred or one hundred and six Other analysts think that it should be higher. My prospect models have one goal in mind. They're trying to predict which players are going to be best during the first three years of their NFL career. To help with that, I measure prospect scorers against the players average of their top two seasons in points per game across their first three years in the league. I know that's hard to follow. Let me just explain. Let's say a running back score is ten, sixteen and fourteen PPERO points per game across his first three years in the league. We'd throw that low score out of ten, and then we would average out the sixteen and the fourteen because those are his top two seasons. He then would have a best two season average of fifteen because the average of sixteen and fourteen is fifteen. That's what my prospect models are solving. They're trying to find players who score best within that metric. It's all laid out in the Prospect Guide. Now, just so you know, that number correlates very very strongly with the player's best season in points per game. So for purposes of keeping this simple, I'm gonna focus more on mac season ppero points per game today, and that's across their first three years in the league, rather than the top two seasons average. It's just easier for everyone to follow. I just wanted to be clear about what my model is solving because it is slightly different. Okay, Now, when you look at speed scores for running backs, you'll find that there is some core between speed score and a player's top season in points per game during his first three years in the league, meaning the higher the speed score, the better that player performed over the first three years of his career. But that correlation has a lot of additional factors at play. One issue that you run into with speed score analysis, and it's more than just speed score, but it's what's called base rate fallacy. Base rate fallacy is essentially when people don't take all relevant data into account within their analysis. One of the best base rate fallacy examples out there is that fifty two percent of car accidents happen within five miles of someone's home. You hear that and you're like, Wow, that's pretty crazy. That's a really high percentage of accidents so close to home. But it's really not that crazy when you consider that the vast majority of driving happens within five miles of people's homes. So the more driving that occurs around your home, the higher the chance for an accident to occur. This is similar to what I see with a lot of fantasy analysis, but specifically, let's focus on speed score analysis. You'll see tables, you'll see charts saying that the majority of RB one seasons in fantasy football are coming from speed scores above one hundred or above one hundred and six, like you're saying in this question, and that's not wrong. Factually, that's correct, But what's your denominator here? There's a base rate fallacy at play because in the end, good running backs are probably gonna be more gifted athletes, and good running backs are generally getting drafted earlier, and they're generally then getting more opportunity, and when they get more opportunity, they score more points. So the more gifted athletes are then scoring more points. But it's not necessarily only because they're gifted athletes. Let's dig into this idea a little bit further. Let's focus on running backs drafted from twenty eleven to twenty twenty, because that allows each running back to have three years of NFL experience. And let's just focus on drafted players to keep this simple, not undrafted ones. So all running backs who are drafted during that timeframe of twenty eleven to twenty twenty percent of the running backs in that sample who hit a speed score above one hundred and six were able to score fifteen or more PPR points per game and at least one season across their first three in the league. So seventeen percent of running backs were very good within the speed score metric, and then they went on to give you a low end RB one high end RB two season in one of their first three seasons in the league. Next, when you look at players with a speed score of one hundred to one hundred and six, that figure drops to about twelve percent, so things are starting to get worse. When you look at speed scores under one hundred, that rate is just seven point six percent. So only seven point six percent of running backs in nor sample with a speed score under one hundred have given us a fifteen plus PPR point per game season across their first three in the league. It seems obvious that speed score matters a lot. Then, right, We've got seventeen percent of players above a one hundred and six speed score to get the fifteen PPR points per game. Then it's twelve percent in the one hundred to one hundred and six range, and then it's just seven to eight percent when that speed score is under one hundred. I mean, again, there is some correlation. It's a weak correlation, but there is some. But you're not getting the full picture when you only focus on speed score and nothing else. You know, that group of running backs with the speed score above one hundred and six, their average draft capital was ninety four, meaning the average player from that group was a top one hundred pick in the NFL draft. And I can guarantee you that they're not being drafted that high because they're just fast for their size. When you move into the next group of running backs to the speed score of one hundred to one hundred and six, the average draft capital falls to one hundred and twenty six, so they're getting drafted over thirty picks later. And then of the backs of the speed score lower than one hundred. Their average draft capital is one hundred and fifty five. If each group had identical draft capital, that would be one thing, but they don't because athletic running backs are generally getting drafted earlier because they're probably more talented. And if a running back is getting drafted earlier, and if that running back is more talented, then he's gonna score more Fantasy points as a result. You see this connection when you only match speed score to Fantasy points. Here's the dirty little secret. You can play this exact same game with other statistics. One metric that's in my model, for example, is reception share. Let's focus on the exact same sample that we did before drafted running backs from twenty eleven to twenty twenty. In that sample, almost twenty percent of running backs with a best season reception share above ten percent ended up scoring fifteen or more PPERO points per game in one of their first three seasons in the league. That twenty percent number, that's a better rate than what we saw with the top speed scores. When you look at running backs who had a best season reception share between five percent and ten percent, that twenty percent rate of getting fifteen or more PPERO points per game drops to just under ten percent, and then for players under a five percent best season reception share you get four percent. So you go from twenty percent in that top group and reception share to ten percent to four percent. Clearly, higher reception share totals matter, and they do, just like higher speed scores are better than lower ones. And the thing is, the draft capital differences between the three groups is very similar to the speed score differences. The guys who had a reception share above ten percent, their average draft capital was one hundred and seventeen. That's a lot worse than what we saw with the top speed scores and the top reception share group, remember, performed better. They hit fifteen plus PPERO points per game at a twenty percent rate. The top speed score guys hit it at a seventeen percent rate. So when you factor in draft capital, you'd have an easier time finding an edge with a reception share than speed score. You can do this with other metrics too. Like I said, just because there's correlation in one does not make it the dominant metric to focus on. It also doesn't make it a metric that you start to filter players out of and That's what's key here. Early declare status is something that gets signal. Yes, players who declare early for the NFL draft are generally better bets than players who don't or running back weight. Running back weight is something I've been talking about a lot over the last week plus on this show. And while running back weight gets signal, even when you adjust for draft capital, it's not enough to eliminate players from your consideration set. It's just an input. And that goes back to the original question here, why does my prospect model look at ninety as a threshold and not one hundred or one hundred and six. Well, while some people might find some significance to one hundred and one hundred and six, they might be solving something different than I am. Early, I looked at running backs who were below a speed score of one hundred and just group them all together, and I said that they get to fifteen PPR points per game at a seven point six percent rate. The thing is, every single running back in that group who got to fifteen PPR points, they had a speed score above ninety, players like Alvin Kamara, Aaron Jones, Kareem Hunt. In fact, according to this data that I'm working with the hit rate of running backs in the ninety to one hundred range, when we're looking at getting to fifteen po points per game across the first three years of their career, it's actually better than the hit rate for players with speed scores of one hundred to one hundred and six. And when you get into the ninety to one hundred range and find players with great receiving profiles, the hit rates get even better than that. This is why the ninety threshold matters. Based on the model, it appears as though when players are under that mark, no matter the production profile, no matter the extra information that we might have on them, it doesn't really matter. But players above that mark, and maybe still below the one hundred mark, if they have a good draft capital, if they have good production metrics, they're still good players. They can make up for a non elite speed score by being great elsewhere, just like a smaller running back can make up for being small by being good elsewhere. That's the difference between my model, which has a ton of different inputs, and testing one singular metric. When you're looking at one metric, you're obviously not capturing all of the other things that a player might be good at and that's no different with reception share. When you analyze reception share as a standalone metric, you're gonna get different results than when you analyze reception share alongside other inputs. You might remember that last season I had questions about Kenneth Walker's production profile. His best season reception share was five point four percent. As I just talked about, we don't see hits from players with that lover reception share very often. But Walker still ended up being a ninety third percentile running back in my model. I ranked him as the RB two in the class because that singular number does not mean everything. Analyzing a running back in such a binary way doesn't make any sense. Walker had a great best season total yards per team play rate. His touchdown share was great, his speed score looked great, his BMI was great, his draft capital was great. Literally everything else in his profile looked awesome. Why would I simply fade a player because of that one thing, But that's what we see all the time. Breakout age is too old, FADOM player is in an early declare FADIM running back didn't have a speed score of one hundred fadom. The fact of the matter is. You can increase your probability of hitting on these guys when you combine all of the things that matter instead of looking at one of the things that matters. I say it all the time, but there's no golden key to fantasy football. There's not one singular metric that you have to abide by when you do your analysis. This is a puzzle and each puzzle piece is a different input. You're not gonna be able to complete that puzzle if you're so focused on just one of those inputs. That's it for today's show, though, Thanks to all of you for listening. If you get subscribed to Late Round Fantasy Football podcast, make sure you are by searching for it pretty much anywhere podcast can be found. You know, for to follow me on Twitter at Late Round QB. If you like some of the stuff that I talked about on today's show and you want to hear more, you should seriously check out the Prospect Guide. You can find it on Lateround dot com. Thanks for listening everyone, I'll talk to you later in the week.