Total QBR has four major flaws.
First, Total QBR is statistical pixie dust that exists only within the walls of ESPN. The network never has published a formula for QBR. Nobody can double-check the data.
“The formula is not something we give out,” said Hofheimer, citing a complex algorithm that takes in many factors beyond passing.
Second, Total QBR is not actually a statistic; that is, it’s not an objective mathematical calculation that reflects performance. Instead, it’s a subjective assessment that betrays human bias. ESPN never has published a formula for Total QBR because one does not exist. Instead, it’s a judgment call.
[........]
Analysts review tape, divvy up credit and assign a score.
That ain’t football. It’s figure skating. And it ain’t statistics. It’s a beauty contest.
[.......]
Third, and most importantly, existing indicators do a much better job explaining individual and team success. Trusty old passer rating is most notable among them. It’s a phenomenal statistic because wins and losses move in virtual lockstep with passer rating.
Teams win when the quarterback passes the ball efficiently. They lose when he doesn’t. The Patriots’ big win is explained perfectly by passer rating.
Brady’s 130.9 rating was 63.9 points better than Weeden’s 67.0 — a surefire sign of a blowout victory. Teams that were plus-50 or better in passer rating margin last year were 51-1 (.981), a number consistent with historic results. The Patriots are 102-6 (.944) when Brady posts a rating of 100-plus.
Passer rating, put another way, has a high correlation to victory. Total QBR has a low correlation to victory.
The Pats’ blowout victory was statistically guaranteed by Brady’s blowout of Weeden in passer rating. But ESPN looked at this guaranteed winning performance and determined that Brady was one of the worst QBs in the NFL this week.
Fourth and finally, ESPN’s indicator makes fatal mistakes common in modern football analysis. It tries to isolate individual “credit” and blame amid a violent swirl of 22 men. It over-emphasizes factors that have zero correlation to success (yards after catch, most notably). And it over-complicates a very simple game: Teams win when they pass the ball more efficiently than opponents. Simple as that. Consider 36 percent of all NFL champions since 1940 (27-of-75) finished No. 1 in passer rating differential. No credit. No blame. Just results.
If ESPN was looking to improve upon existing stats, it failed miserably. This week’s rankings, meanwhile, betray either a bad stat or bias against Brady inside the halls of ESPN.