Rewind: Predictive Data

00:00:00
Speaker 1: Hey everyone, JJ here with another Late Round podcast rewind. Last year, I sat down with Josh Hermsmeier of five point thirty eight to talk about predictive data. We've got all these numbers in the fantasy football world, but a lot of those numbers don't have a whole lot of meaning. What types of numbers do, what's important? And how do we adjust our analysis as a result. Josh helped answer all of these questions last March. I hope that you enjoy the interview, and I'll talk to you on Friday with the Weekly Mailbag episode. All right, Josh, so airyards dot com and the work that you've done with aeryards as a statistic it's become a really big piece to the fantasy football community. Can you talk about what led you to creating that site and why aeryards have become such a focus.

00:00:42
Speaker 2: I guess I created it because I, first of all, I found the data and that was just really fortuitous. I was digging through kind of public databases and Python code and r code that was able to like at least dip their toe into this game center yeah that the NFL has on their website, and it was really urcane and none of it was documented very well, and in any event, it led me to kind of digging in a little further. And then I found out that the NFL was actually publishing you know, incomplete air yards. And we always had complete air yards. Is all you need to do is subtrack yak from seving yards to get that. But the piece that we were missing was this incomplete part. The reason why that's important is because it's like this notion of like what was intended to happen and didn't happen. And so anyway, I found that data, and you know, so many, so many things when it comes to football is just having the data. I mean, I don't think we have a problem with analysts. I think we have we have some of the best. We just have a problem with data. And so I was fortunate to get my hands on it. I knew it was important when I did a correlation with pffs a dot and it was like point ninety nine, So I knew I had something good, solid data. And then and then really I just went back and read what others smart people had done with depth of target, like Mike Clay, and he kind of collapsed the data into one number like that's what AIDA did, is the average depth of target. And then what I did was instead was spread it out and I said, let's look at every depth and see how players perform. And so yeah, So after I did that, I thought, you know, this is really interesting. I'm going to make a website. And I did. And that's probably the most kind of fulfilling part of the whole thing is that actually gets used. That really is super cool from my preserative, and it makes me really happy. Yeah.

00:02:30
Speaker 1: I mean I use it all the time. I sided on this show all the time. It's a very very useful resource. Can you explain sort of some of the metrics that you've created that stem from air yards? I know that you've created a few, sort of talk about why those statistics have been so important in your process and whether it's real football or fantasy football analysis.

00:02:48
Speaker 2: Sure, yeah, and it all starts with fantasy I think because it's important to me, because it's important to me, like it's my life as the stuff I choose to spend my time on or my fantasy leagues. And I think it's the same saying with other folks. I think, you know, you can help them where they live. First, that's the most important thing that they respond to. So anyway I think people will run away from fantasy is a huge mistake. But it does help in real football because when you adjust for depth of target with something like pacer or racer pacers, basically you can think of it as a dot over or excuse me, yards per target over a dot for a quarterback, same kind of thing for a wide receiver, and it's basically how often he converts a yard in the air into a yard on the ground, whether it's through the catch or the yack after, and so really just kind of it's a new way of measuring efficiency. It's stickier, in other words, more predictive than yards per target as an efficiency metric. So in the realm of efficiency metrics, it's the best that I found. And then for quarterbacks, adjusting for depth target turns out to be hugely important, you know, I mean, it's really intuitive, but it is the case that you know, a fifteen yard deep out is harder to complete than a screen pass, so adjusting for that is gets you closer to what a quarterbacks real performance level is.

00:04:02
Speaker 1: Right, So you mentioned the notion of stickiness, which I think a lot of fantasy managers are pretty interested in, you know, especially those who listen to this show in particular, what type of data have you found that fantasy managers tend to sort of overrate when they're projecting an upcoming season. So we're you know, we're really talking about statistics that aren't sticky.

00:04:22
Speaker 2: Yeah, mostly efficiency metrics. Unfortunately, I think we tend to view football volume the same way maybe certain people view like maybe baseball volume where it's kind of empty. It's an empty metric. It's like, you know, it doesn't really tell you as much as it does in football, and football, if you get a target, that means they I mean, look, they're not going to a team, a QB, a coaching staff, a play caller is not going to waste it down on a scrub. So the more targets a guy gets, the more indicative that is of his talent, his true talent, team's view of his true talent. And then when you leverage those targets and put how many yards are behind them, which is what air yards is, you can get at the true value of that guy in the offense. And so that's what whopper is is this weighted opportunity rating. It takes those two things, terms them into share metrics and then weights them appropriately basically two to one, twice as much weight on targets as air yards, and you get this metric that's really really good at telling you who's likely to be good next year. And so yeah, I think But the main thing that folks that I talk to because you see a guy and you see high efficiency, right, and you think that is evidence of him being a good football player. And I think we can overweight that. Sometimes I don't say throw it all the way out, but I think we can overweight it.

00:05:39
Speaker 3: Yeah.

00:05:39
Speaker 1: So I mean something that I always talk about on the show is that targets are skill statistic. I mean, you have to be open in order to get those targets in the first place. That is that a reason This is kind of a tangent, but is that a reason why you are a Jarvis Landry truther?

00:05:53
Speaker 3: Yeah?

00:05:53
Speaker 2: I mean absolutely because and not only that, but he was efficient on his own terms. So like if you look other players who got the types of targets he got, he.

00:06:03
Speaker 3: Was hugely efficient compared to them.

00:06:04
Speaker 2: So, I mean the argument against Jarvis Landry, which I am totally sympathetic to, is that his targets, or at least before last year, weren't as valuable as targets to say ab or you know, other really good wide receivers. And yes, yeah, absolutely don't don't argue with that. But his targets are way better than a target to a running back. They're way better than a handoff, for God's sake. So I mean, let's not, let's not, let's not like throw the baby out with the bathwater, you know, baby steps to something highly efficient in the NFL is more than welcome in my opinion.

00:06:36
Speaker 1: Yeah, absolutely so is you know, on the flip side, is there any data that folks tend to underrate or not pay enough attention to. I know you mentioned volume, and obviously volume is stickier than than than most efficiency stats, But is there anything kind of more in depth than just simply you know, targets or attempts.

00:06:53
Speaker 2: You know, everyone's so super sharp at this point. I think, I think really the way to kind of leverage you know, data and statistics is to is to just wait the opportunity properly. I really don't see any other edges. No, there isn't really anything everyone's everyone's putting. Probably now the proper weight on the leve We'll talk about this later, but things like you know, you know, athletic measurements, things like efficiency, maybe they're overweighting too much, so you might have an edge there a little bit. But really, the only like repeatable edge I see in fantasy is in dynasty leagues, where in a startup you are not averse to rostering older guys because there's this really huge bias for young players.

00:07:36
Speaker 3: Yeah.

00:07:36
Speaker 1: Yeah, people, people seem to think that they know a lot more than they actually do. It's sort of sort of the approach.

00:07:41
Speaker 3: That I take.

00:07:42
Speaker 1: So one of the issues that I've always had with testing the significance of certain metrics in football is that almost everything just shows little correlation in this game, especially if you compare it to another industry. Let's say, so, you know, some metrics and statistics definitely are bad objectively, but if you found that looking sort of at the extremes because of these low correlations, looking at the extremes, can that be helpful? And are there any examples of this? You know, one thing that usually pops in my head when I think about this is the NFL combine. You know, in some it's really hard to find significance with individual workout drills, but there are a lot of layers to it. Since we know that the combine can help a player's draft stock, you know, it might be able to help immediate volume in the NFL then, and it seems that that we can ignore sort of these outliers like or at least, you know, at least put some weight to these outliers like we saw with Elijah Holyfield running a four seven eight, because we know historically running backs who run that slowly just haven't done anything in the NFL. So how do you sort of combat this as someone who's testing the significance of data all the time. Do you just ignore most data or are you looking closely at those outliers.

00:08:52
Speaker 2: So it's a difficult question to answer because really teasing apart when you put all that stuff that you just mentioned into a model that includes things like past production, it's really hard to tease apart what's actually helping, especially if you use something really like exotic, like a machine learning model that's using all sorts of like submodels inside of it and actually getting to what's what's really driving this. You know, it almost always comes back volume opportunity that and so and the other minor things are really hard to like rank. But I would say that, you know, for skill position players, for fantasy relevant players, things like speed can be predictive, Like, for instance, for guys that are gonna be stretch x is forty times, it does actually correlate really well with on field game speed that's measured by the NFL. And so if you want a guy like John Brown or something like that, who you're going to use as a stretch X and maybe he'll get those shot plays and maybe he will be a guy that profiles as someone you play in DFS for instance, and might have a big week, then that that's actually something that's worthwhile to look at. But really, I think the combine and and numbers that are generated there are are way more useful for non skill position players and edge rushers and offensive linemen and things like that, but less less important for fantasy.

00:10:10
Speaker 1: Yeah, I've sort of found the same thing. I mean, I have a pretty basic prospect model that I run through throughout, you know, once the NFL season's over, and I've just found that college production is just far far more predictive than simply throwing in all these agility scores and to trying to find something through that. But again, you know, I've I've was just curious because you're constantly testing these specific statistics and you're finding great I mean, there are great findings as a result, and you're helping the community move forward with a lot of this stuff, and I was just curious if you, like, let's just say, for an instance, let's just use this Elijah holy Field example. If Elijah Holyfield runs a four to seven eight like he did, do you see that as a legitimate red flags? As you know, I'm never gonna draft this guy because he's probably not going to be good, like strictly based on that forty time.

00:10:59
Speaker 3: Yeah, I think thresholds matter for sure.

00:11:01
Speaker 2: And I can also tell you talking to folks that actually work for teams that all those combined metrics are in their models. So I mean it's not the case again that like you know, people are ignoring it completely. I just can't answer with any real degree of certitude about what level of importance they have relative to other more important things that we know are important. But yeah, no, thresholds matter. Like we talked about Kyler Murray all this off season. You know, five ten, you know he's too short. He's too short. There is certainly a limit to how tall you have to be to play NFL quarterback. That I mean, there's no doubt about it.

00:11:40
Speaker 3: Is it five to ten though?

00:11:42
Speaker 2: I mean I don't maybe I don't think so, but probably five to six that you're not going to be a good quarterback there. And so I'm certain that if you ran a five flat that you're not going to be good in the NFL. So, again, is Elijah Hooleyfield? Does you have other trades talents? Will you be in a better situation asked to do a secific thing that can make up for that?

00:12:02
Speaker 3: I don't know.

00:12:02
Speaker 2: It's probably unlikely, but I think I think it takes better analysts than me to make those kind of determinations.

00:12:11
Speaker 1: Yeah, I feel you. I feel you speaking the NFL draft, that's the focus this time of year. I remember you doing some interesting work last year with running back prospecting, especially. I remember you know you were into I was into Ashod Penny as well, but you were looking at specific statistics that you found to be fairly sticky with with running backs that were jumping from from the from college to the pros. Can you talk about that.

00:12:34
Speaker 2: A little bit. Yeah, I think it was a combination of touches and broken tackles. And I looked at like yards before contact, yards after contact, all that stuff.

00:12:44
Speaker 3: Neither of those were helpful.

00:12:46
Speaker 2: It was the simple fact of somehow eluding or breaking a tackle through physical violence. You know, like those things seem to be the traits that we pick up in the data that are actually carry forward and and translate into the NFL. Guys like Cream Hunt and you know, we have lots of success stories with this. It's you know, if you can break a tackle, if you can stone a guy and then keep moving, you know that that will translate to the NFL. And then if you're athletic enough to even be invited to the combine, you know, now now you put yourself into a special category crazy or show a penny. I mean, it really was the case that when he did get some volume, he looked pretty good.

00:13:25
Speaker 1: Yeah. I totally agree with you, totally agree.

00:13:28
Speaker 2: So I don't think he's like a bust in terms of talent, but it is obviously the case that you can find guys with his equivalent talent in the sixth and seventh round, and that's what Carson Is. So, you know, and I think that just goes to, you know, a topic that near and dear to your heart that you don't want to talk about it anymore, just about the fundatillit running back.

00:13:46
Speaker 1: Yes, yes, I've kind of I've kind of retired from that, at least at least on Twitter. I can't. I can't spend the hours and hours of the rabbit holes that you can go down and talking to people about that topic. So, Josh, you wrote a really really awesome piece on five point thirty eight about a week ago that was called the NFL is drafting quarterbacks all wrong? And in it you found that completion percentage seems to predict NFL success fairly well. But you took that a step further and you found that the difference between completion percentage and expected completion percentage is even better. So, you know, I know that it's a long study. It's a really really great read. I recommend anyone listening to this to go read that. But can you sort of walk through what you discovered and is this same metric meaningful with quarterbacks currently in the NFL?

00:14:29
Speaker 3: Yes, happily yes. So to the last part.

00:14:32
Speaker 2: First, you can use that exact same analysis on current NFL qbs and find similar results in terms of correlation with yards per target, in terms of EPA per play, all the things we care about. Completion percentage over expected is a better predictor than anything else we have. So that's great because and the reason why that's great, and this is really what I talk about in the article, is because I think completion percentage over expected is capturing things that aren't exactly related to on field play. It has to do with what's between the ears, and that's the hardest thing to get at right. It's like this idea of mental processing football IQ, throwing with anticipation. These are things that I think captured in that metric. I mean, can't prove it, but I believe they are. And so for me, like what's exciting about it is it's it's pushing me towards some future work that I've been discussing with some folks that I hope to publish soon that we'll kind of get even further at that idea of what's between the ears of the quarterback, because I think that's what's missing from NFL analytics, because getting the quarterback right is so important and if you have the wrong quarterback, it may be the case that we are the NFL talent evaluator is actually better at the rest of the positions than we think. But we put them on teams with bad quarterbacks, and everyone suffers, you know, And so I think we get quarterbacks right, maybe these other things improve as well. And anyway, my whole, my whole thing with it was that completion percentage.

00:15:54
Speaker 3: Guys like fifteen.

00:15:55
Speaker 2: Eighteen years ago discovered that it was a translatable trait, and I just say, maybe we can make it better. I said, you know, if we adjust for the depth of the throw, maybe that helps. And then if we adjust for the talent level around them and their conference, maybe that'll help.

00:16:09
Speaker 3: I did both those things and it turned out that it did help.

00:16:12
Speaker 2: And then I built a simple model just kind of predicting which of this upcoming class maybe the best bets to be good quarterbacks. Kyler Murray top of the list. There's a surprising Willgrew number two. Haskins was fourth or fifth, so and all these were like they had better than fifty percent shot of being above average NFL quarterbacks. So these are pretty good qbs in terms of prospects. Which was interesting because people said this was a garbage class coming in and Murray actually looks like one of the better ones in the past three or four years.

00:16:42
Speaker 1: So yeah, I thought it was an incredibly interesting piece. Definitely when that people should check out.

00:16:47
Speaker 2: Josh.

00:16:47
Speaker 1: Thank you so much for hopping on and talking a little predictive data with me today. If any of you aren't following Josh on Twitter, you can find him at Frisco Josh and I'm of course on Twitter at Late Round QB. And make sure you're subscribe to this podcast be searching for it pretty much anywhere podcasts can be found. Thanks for listening, and I'll be back in your ears Friday with the Weekly Mailbag episode.