NFL forecasters vs.  evaluators: Analyzing the differences between NFL Draft prospects

NFL forecasters vs. evaluators: Analyzing the differences between NFL Draft prospects

We’ve been putting together a myriad of insights that come from the project to combine player rankings from experts across the industry, including figuring out who the most polarizing players are in the 2022 NFL Draft, which players might be over- or under-drafted against the consensus board and what insights can be generated just from highlighting the board’s disagreements with The Athletic‘s Dane Brugler.

But something that can be overlooked is how experts put together their rankings. All NFL Draft analysts have their own processes for incorporating information and determining which pieces of data to use or discard when writing their scouting reports and constructing their player rankings. Some use broadcast footage from college games and write their reports with only that perspective. Others have access to coaches’ film with more viewing angles. Some travel the country, like some NFL team scouts, watching games live and going through the libraries of college tape to write their reports.

Other analysts incorporate statistical data from the most recent research available on how to project player performance to the NFL, whether that involves wide receiver “market share” or combine workout metrics.

But that often isn’t enough to tell the complete story of a prospect. Even NFL teams don’t think so. After watching film of a player, they will ask questions of coaches, trainers, teammates, equipment managers and more to get a more complete understanding of a player’s temperament. They check police reports to see if a player has had any run-ins with the law. They will ask medical questions of the college training staff and supplement that with the medical reports produced by their own trainers during the NFL combine, when prospects undergo a medical evaluation.

Third-party analysts can have access to some of this information, as well. Sometimes that’s directly through the relationships they’ve built over the years at various colleges, while other times it can be indirectly, through scouts and NFL teams who uncovered the data.

The ability to learn about the NFL’s perspective on a player, whether it’s from a scout’s assessment of a player’s ability or important information about a recent injury, will color an analyst’s perspective, unconsciously or not. And that divide in information can be pretty sharp.

Every year I’ve done this big-board project, I’ve been able to divide the two types of big boards between those of forecasters and those of evaluators.

Forecasters tends to have big boards where the rankings are tightly clustered together and come from analysts who have built up relationships and sources within colleges and NFL teams. They can more easily respond to previously unknown information about a player’s character or injury history. They might wonder why a player isn’t producing and receive an answer from an offensive coordinator about how that player’s role limited their production but not their talent.

But this information can also be subject to the kind of jockeying that teams will often do before the draft. Perhaps an NFL team member is only telling partial truths or obscuring important context when discussing a player in the hopes that the player falls to his team. Or it could be the case that a college without many draft prospects in the past tries to boost the draft stock of its best candidates to help the school’s recruiting efforts.

Draft analysts have to filter out that motive and information when building their database of player grades and will supplement those grades judiciously with the information they receive. There are pros and cons to that approach.

On the other hand, evaluators are much more likely to use game tape and analytical data to construct their boards, without much input from scouts or colleges. They may not have critical information like what a player’s assignment was on a specific play or how a player is recovering from an injury and will have to rely on second- and third-hand accounts from the media to construct a more accurate board.

Rankings from evaluator boards tend to be significantly more varied, without as much clustering of player rankings, and there tend to be very few boards from that second category that follow the strongly correlated rankings of the forecasters.

Despite the name, forecasters aren’t attempting to predict the draft with their player rankings — that’s what mock drafts are for. They’re called forecasters because it’s inevitable that the information they receive will impact what they do and they just so happen to beat out the evaluators much more often than not when it comes to predicting the draft order of players.

Knowing that these two approaches produce different rankings and different boards, perhaps we can glean something from the differences. Which players in this year’s draft did the evaluators and forecasters disagree on the most?

Forecasters vs. Evaluators

RANK

PLAYER

SCHOOL

POS

FC RK

EV RK

130

jerome ford

RB

182

121

149

Tyler Badie

RB

187

136

119

Justin Ross

WR

170

117

95

Cam Jurgens

OC

66

99

147

Bailey Zappe

QB

190

139

128

Dominique Robinson

Ed

96

134

4

Kyle Hamilton

S

6

2

139

Romeo Doubs

WR

172

131

118

Rachaad White

RB

148

110

145

Damarri Mathis

CB

121

154

84

David Bell

WR

108

79

133

JT Woods

S

109

140

93

Marcus Jones

RCN

73

97

75

Kingsley Enagbare

Ed

94

69

51

quay walker

LB

39

54

The forecasters are higher on six of the 15 players that have drawn the biggest disagreements between the two boards. The most common theme here is that these players are all incredible athletes and some of the top at their position. If this gives us an insight into the NFL thinking, it could be that this year there just aren’t very many polished players and that they would rather coach up the players with the highest ceilings.

The biggest disagreement for the forecasters seems to be Cam Jurgens, a center from Nebraska. They rank him 33 spots higher than the evaluators do and seem optimistic about his development path. The fact that he was initially recruited as a tight end and had to learn to play center quickly seems to appeal to NFL evaluators. There does seem to be a slight bias in forecaster evaluations for run-blocking prowess for offensive linemen over pass-blocking capability, and this could play a role as well.

Dominique Robinson is another position converted. He arrived at Miami of Ohio as a quarterback prospect before transitioning to receiver, where he played for three years before switching to edge defender in his final two years. As a novice at the position, it’s understandable why he doesn’t have much production coming off the edge: two sacks in 2020 and 4.5 in 2021.

More athlete than player, the NFL must view his coachability and athletic traits as unique assets. While generally true across positions, the forecaster preference for super-athletes seems particularly pronounced at edge defender, where Amare Barno and Travon Walker are good examples of that prototype.

Defensive backs Damarri Mathis of Pitt, JT Woods of Baylor and Marcus Jones of Houston are interesting cases, as well. All three are remarkable athletes for their position, and Jones might have joined Woods and Mathis in running sub-4.40 second 40-yard dashes had he been able to run at his pro day or the combine. Jones will likely be relegated to a slot role because of his 5-foot-8 frame, which could play a role in the disagreement, too.

Forecasters also happened to like Quay Walker, a linebacker from Georgia. Essentially a sub-package player until his final year, Walker follows the theme of this list, showcasing athleticism without much of a history of production. Analysts seem to think there’s a good chance that Walker will have a much more productive NFL career than college career.

The player who stands out the most for the evaluators is Kyle Hamilton. His 40-yard-dash time — run twice at the combine and at least once at his pro day — is concerning, and it’s difficult to figure out why a top prospect would run so poorly so often. Evaluators have largely dismissed this finding, but, curiously, forecasters haven’t. The difference in the NFL draft between the second slot and the sixth slot is enormous, so seeing such a large disagreement for an otherwise top player is notable.

Three of the players evaluators preferred over forecasters are running backs. In this case, these running backs tend to have already filled out their frame and can bring a little bit of power to bear, at least more than most backs can at their respective sizes. Also worth noting is that all three backs also produced at least 1,300 yards on the ground in their final season — something not necessarily common among the running backs outside of the top three.

There’s always at least one player with a big injury question who ends up on the list, and Justyn Ross from Clemson is the perfect example. Ross had a phenomenal freshman year at Clemson before injuries and a big surgery derailed his path. Whether he can recover his previous athletic ability is the primary question surrounding him, and it seems like forecasters are more pessimistic than evaluators on that front.

Romeo Doubs from Nevada and David Bell from Purdue also both make the list at receiver. Highly productive in their careers, there are concerns about their ability to translate their success to the next level. For Bell, that means overcoming a lack of athleticism. For Doubs, it’s physicality and nuance.

That bias for production among evaluators extends to Bailey Zappe, one of the most productive quarterbacks in NCAA history. Finishing his final season with nearly 6,000 passing yards and 62 touchdowns, Zappe is an easy quarterback to fall in love with. Whether his traits translate — including his athleticism and arm strength — is another question, and it seems like evaluators are more optimistic than forecasters.

Kingsley Enagbare from South Carolina is another player whose medicals are vitally important. He was impressive to watch in the SEC, but a hip surgery that left him unable to walk for two months after his junior year will be something the NFL notes, along with some measly production numbers. Forecasters being low on him may suggest that his medicals haven’t checked out.

Overall, it seems like forecasters almost universally preferred raw athletes over proven producers. Last year, the trend ran a bit in the opposite direction: Evaluators preferred athleticism and upside over production and technical skill.

Over the years, I’ve found that these differences do a better job of giving us context for how these rankings are achieved. In almost every year — 2020 stands out as a glaring exception — the forecaster board has done a better job of predicting the draft slot in which a player goes. But over the years, neither board has been substantially better at predicting player outcomes, suggesting that there are some opportunities for NFL teams that strategically move away from the forecaster consensus.

(Photo of Quay Walker: Joe Robbins/Icon Sportswire via Getty Images)

.

Leave a Comment

Your email address will not be published.