ESPN QBR v. FO DYAR

[TheSolderKing]

D (Major Phuck up) FINAL ANSWER




Where is Cam Newton?

 

[Triple-T]

QBR is dumb.

The problem is the subjectivity on player contribution, but even more so, the clutch index is likely the biggest culprit. You can't penalize a QB for being good early and not having a tight game and then reward another for not doing well early. If I only care about the win, I want the biggest lead I can get, as early as I can get it, to put the game out of reach. Every point matters regardless of when it is scored. QBR perverts that.

 

[The-Hooded-One]

Cam Newton? He's busy throwing interceptions. You can't seriously believe 1TD-3INT belongs in the top ten.

 

[TheSolderKing]

Cam Newton? He's busy throwing interceptions. You can't seriously believe 1TD-3INT belongs in the top ten.

but he threw for 4 TD's and 400 plus yards


4-3 TD/INT Ratio it certainly is NOT a negative ratio

 

[Deus Irae]

but he threw for 4 TD's and 400 plus yards


4-3 TD/INT Ratio

He's got 3 TDs and 4 INTs on the season.

 

[The-Hooded-One]

but he threw for 4 TD's and 400 plus yards


4-3 TD/INT Ratio it certainly is NOT a negative ratio

These lists are for week 2.

 

[Anteros]

I like the traditional NFL passer rating best... perhaps it is familiarity..

NFL Stats: by Player Category

Yup, I vote for this one also. So "none of the above" to the poll picks.

 

[BradyBranch39]

Anyone want to guess the length of T.Jackson's second-longest completion this season?

edit: just tweeted the answer

 

[lamafist]

I'd like to read it. Layman's terms suck. I'd like to know what he's actually doing. Do you have a good link?

It's on espn with the rest of the QBR stuff. I think the FAQ even links to it. He's still clearing writing for a general ESPN audience, so he doesn't get to indulge in the wonk side like the guys at Football Outsiders or Advanced NFL Stats, but he does go into more detail about how they use the charting data that people worry is subjective. He explains how the potentially debatable 'judgement calls' are used less to affect an individual QB's score directly, but rather as collective data to establish baselines for how much a certain type of pass's completion rate varies based on QB as opposed to WR -- like, hypothetically, if passes to RBs in the flat on 1st and 10 show more variance from QB to QB and less variance based on WR than other types of passes, that informs how much weight to give increases in expected point values in terms of the QB's contribution.

 

[lamafist]

QBR is dumb.

The problem is the subjectivity on player contribution, but even more so, the clutch index is likely the biggest culprit. You can't penalize a QB for being good early and not having a tight game and then reward another for not doing well early. If I only care about the win, I want the biggest lead I can get, as early as I can get it, to put the game out of reach. Every point matters regardless of when it is scored. QBR perverts that.

That's not really how the 'clutch index' actually works, though I can see how you'd think that.

This is another frustrating situation where ESPN's poor choice of terminology screwed things up for them. Calling something a "clutch index" sounds like something that specially factors in last-minute heroics. Rather, what Dean Oliver's doing is a play's net change in expected win probability to weight the basic yardstick of QBR, which is the change in expected points probability.

Over at Advanced NFL Stats, Brian Burke keeps graphs of the fluctuation in win probability from every play of every game during the season.

Basically, the way it works is that, using the outcome of every football game in modern NFL history as a sample pool, you come up with a win percentage for teams that have been in the same situation -- field position, score differential, time remaining -- as a given team on a given play. After the next play, you re-calculate the historical win percentage based on the new game situation. The difference between these situations gives you an idea of how much that last play affected a teams' likelihood of winning.

Now, while time remaining does enhance win% volatility, score differential is the more prominent factor. If you look at the WP chart for the game against the Chargers last Sunday, you'll see that the Pats did most of their win% increasing in the 2nd quarter. Brady's contribution in expected points during that stretch will benefit from added weight due to its high "clutch index" -- it's where the most game-deciding plays occurred.

Assuming a fixed number of points a team's defense will ultimately give up, since we're talking probability, every game has the same potential for net "clutch index" weight -- after all, until the games over, a team's win % will be somewhere between 0 and 1. This is why "clutch index" is such a poor choice for what it reflects in the QBR metric -- it adds weight to the QB's performance when the game was being decided, whether that was in the first half or last play.

 

[lamafist]

This. I'm used to it, I like it better, and under most cases it seems accurately place the top QB's in order more often than not.

The QBR is dumb and uses opinions based on what someone thinks is important or not.

Actually, you've got it kind of backwards.

Traditional passer rating is based on what someone just decided was important or not. The relative weights of completion %, yards per attempt, TD% and INT% are actually entirely arbitrary.

In passer rating, completion % is reduced by .3 and then multiplied by 5 before being factored in. YPA is reduced by 3 and then multiplied by 0.25 before being factored in. Why? Because. That's why.

QBR, at least, is based on actual measurable contributions -- viz. how much did completing that pass from the 20 to the 30 yard line on 3rd down increase the average number of points scored by a team in that situation, weighted by how much that change in expected points is likely to affect who wins the game.

 

[Deus Irae]

Actually, you've got it kind of backwards.

Traditional passer rating is based on what someone just decided was important or not. The relative weights of completion %, yards per attempt, TD% and INT% are actually entirely arbitrary.

In passer rating, completion % is reduced by .3 and then multiplied by 5 before being factored in. YPA is reduced by 3 and then multiplied by 0.25 before being factored in. Why? Because. That's why.

QBR, at least, is based on actual measurable contributions -- viz. how much did completing that pass from the 20 to the 30 yard line on 3rd down increase the average number of points scored by a team in that situation, weighted by how much that change in expected points is likely to affect who wins the game.

They are both based on what someone decided was important or not, and QBR failed in its very first week.

 

[lamafist]

They are both based on what someone decided was important or not, and QBR failed in its very first week.

Nobody had to "decide" that increasing the number of points a team is likely to score or the probability they will win the game is important. (Well, I suppose, in the greater scheme of things, football isn't all that important, but, you know...)

Whatever one's complaints about QBR, that it is built on far more sound fundamental principles than traditional passer rating is objectively true.

Also, I'm curious what you mean when you say a metric "failed." I'm not sure what that could possibly mean.

 

[Deus Irae]

Nobody had to "decide" that increasing the number of points a team is likely to score or the probability they will win the game is important. (Well, I suppose, in the greater scheme of things, football isn't all that important, but, you know...)

Whatever one's complaints about QBR, that it is built on far more sound fundamental principles than traditional passer rating is objectively true.

Also, I'm curious what you mean when you say a metric "failed." I'm not sure what that could possibly mean.

You're asserting an opinion as fact. The QBR does, indeed, base itself on what someone finds important. You can explain why you think that the right things were used, but you can't honestly make the rest of your claim.

And it failed because it didn't produce the best player at #1. If it can't do that, there's no sense in trying to change from the old system.

 

[lamafist]

You're asserting an opinion as fact. The QBR does, indeed, base itself on what someone finds important. You can explain why you think that the right things were used, but you can't honestly make the rest of your claim.

And it failed because it didn't produce the best player at #1. If it can't do that, there's no sense in trying to change from the old system.

It's not a claim. It's a statement of fact. QBR is based on two things that are not only objectively but tautologically important in football: points and wins. This is why it's a superior stat to passer rating -- it starts with what's by definition important, and works back from there, rather than starting with component like completions, yards, TDs, INTs, etc.

A team that startss off 1st and 10 on their own 20 has a certain number of net expected points calculated based on the outcomes of every team that's ever been in that situation. If the team runs a passing play, gains 15 yards, and now has 1st and 10 at their own 35, they now have a different, greater, number of expected points based on the outcomes of every time a team has had a 1st and ten at their own 35.

QBR isn't interested in the 15 yards gained -- that's a component of indeterminate value. All it cares about is the relative difference in probability of scoring points from the team's circumstances before the play, and after the play, because this is, by definition, what is important about that play.

Then it factors in winning. A change in the likelihood of winning is a function of the change in point expectations. Again based on every time teams have been in these relative situations -- now including point differential and time remaining -- you can calculate how much this change in expected points changes the likelihood that the team is going to win.

These are the two things that QBR measures -- points and wins -- and everything else breaks down to how to effectively isolate the QB's contribution to these two objectively important data points. That's where things get dicey -- and it has nothing to do with anybody deciding something is important or not.

As for the supposed "failure" of the metric -- are you really arguing that a statistic that doesn't simply parrot our preconceptions back to us has "failed?" I'm think you might not understand the point of applied mathematics.