No, they're coming up with pretty much the same thing. A slight difference is insignificant, and the terrible skewing it's done with Smith's rating is actually a good example of it being a complete failure as a measurement, not as an example of a success.
...
"The more tools the better" is only really applicable if the tools work and have different purposes. Short of that, having enough is having enough. You don't need to keep 10 identical circular saws if you're just one person doing small projects.
In that same vein, when you're pimping your shiny new QBR, it should be better than what it's theoretically replacing. Unfortunately, ESPN's QBR sucks, because it's failed repeatedly at precisely the things it was supposed to be improving upon.[/QUOTE]
You seem to be contradicting yourself. Is there only a "slight difference" between QBR or is there "terrible skewing" going on? Oh, and on what are you basing your contention that QBR's relatively low ranking of Alex Smith is less indicative of his contribution to the 49ers offense than traditional passer rating? You seem to be basing your notion of a metrics "success" or "failure" based on its ability to produce results that conform to your preconceived "gut" sense of how things should rank out. With that reasoning, what's would be the point of any statistical metric at all?
All statistical metrics tell us
something. For sure, yards per carry doesn't tell us the whole story of a running back's performance, but it's also not subject to natural cognitive failures like the recency and primacy effects, base rate neglect, anchoring bias, availability heuristic or confirmation bias. So while it could never (and should never) replace your eyeball judgement, if you are able to recognize the particular susceptibilities of each evaluation method, you can synthesize a greater more rounded understanding.
As for QBR vs. Passer Rating, there really should be no competition. The
only thing passer rating has going for it is that it came first. Other than that, it's a pretty ridiculous and meaningless construct.
The only four things it takes into account at completion percentage, yards per attempt, and TD and INT percentages. Why these? Because they were the only stats tracked at the time. (Sacks weren't tracked, so throwing the ball away is worse for a QB's passer rating than not feeling backside pressure and getting strip-sacked.) The biggest problem is that, while we can all agree that all four components of passer rating generally indicate a positive effect on a team's likelihood of scoring and winning, passer rating doesn't even attempt to answer the question of their relative weights. In passer rating the importance of completion percentage vs. YPA is
completely arbitrary.
Worse yet, the way each component is calculated is normalized to the statistical averages from when it was devised in the 70's. The idea was that an average performance gave you 1 point in the base calculations, and tying a record gave you 2 points, and these numbers were combined and adjusted so that a good performance would be somewhere around 100. So, since at that time, the average completion percentage was 50%, and the season record was 70%, completing 50% of your passes gave you 1 point, and 70% gave you 2 points. Let alone the obvious problems of the fact that these averages have changed, there's also the arbitrary assumption that a record-tying performance was exactly twice as valuable as an average one.
Ultimately, the passer rating stat can't be said to really be a measure of anything, which makes it pretty much useless for anything but decoration. At least something like yards per attempt is a direct measurement of something, not an arbitrary amalgamation of numbers with no thought put in how and why they're combined.
QBR attempts to provide a metric that has the expanded scope of a passer rating but remains a true, weighted, measure of something specific. At its base, QBR is a measurement of a quarterbacks per-play effect on his team's Expected Points values. This is nice, because it avoids the question of how important something like a QB's completion percentage is to scoring touchdowns. Expected Points Added goes directly to the question of how much did that play increase or decrease the amount of points the team is likely to score on that drive.
The next step is to attempt to weight this number according to how much the quarterback had to do with the change in expected points and how much had to do with the receiver or chance. This is where ESPN's resources proved invaluable. They had the manpower to have people go back and track details about plays not included in the play-by-play so that they could then figure out how much these things varied based on different quarterbacks, and this figure was used to 'weight' how much of the EPA the QB is credited or debited with.
For example, the degree to which the yards gained by a completion to a back in the flat varied from QB to QB vs. from RB to RB over the last 5 years is used to determine how much 'credit' for those yards the QB gets. Meanwhile, a QB is likely to get most of the negative EPA for a pick six on a quick out, but not on a pick six thrown to Ed Reed 25 yards downfield.
So now that the QB's EPA is weighted for his position's average contribution to the outcome, it's further weighted based on the difference in Winning Probability before and after the play. This was stupidly called "Clutch Index," but what it really does is provide a smooth gradient with which to weight every change in EPA based on its likelihood of affecting the outcome of the game. Since this is based on historical probability averages, not only is there nothing arbitrary being done, but it also naturally accounts for the relative importance of situations we might not be 100% aware of. For example, if cutting the lead to one score just before the end of the half has historically had a disproportional effect on the team's likelihood of winning, a weighting based on Win Probability Added will factor that in without anyone having to make any decisions about it.
So in the end, unlike passer rating, which can't be said to be a measure of anything, QBR can be neatly summarized as a percentile function of the number of points above the expected league average your QB contributed to, weighted for positional responsibility for the outcome and leverage value.
So not only are there compelling reasons to think it should be more accurate than the arbitrary frankenstein's number that is passer rating, but it also creates a versatile framework for other statistician to build on.
