I'm currently having an argument with ProFootballFocus on Twitter about Brady.
Saying he had a +1.1 game vs Chicago which is just above an average QB game.
Why? Because he should have been picked off a few times.
Ridiculous. Rivers should have been picked off at least 4 times last night but it was dropped.
And like Brady, he recovered from the poor throws immediately to score a TD.
Their QB ratings are complete garbage, and they don't seem to care they they correlate with nothing of interest to anyone but themselves. Owslek has pointed out how their system is oblivious to the value of good decision making, which is inane when evaluating QB play. I pointed out to them that their system actually scores QB's higher for making decisions that decrease the likelihood of winning. Utter insanity.
Since they use broadcast tape for their evaluations, their ratings of anything in the passing game except pass protection are nearly useless as they're necessarily incomplete in an unknowable way.
It's only with OL, DL, and LB play against the run that they have enough information to even attempt to fairly score player's performance. They are at least potentially useful there, especially as statistical scoring of those very players is meaningless -- unlike with skill position players.
Even for those "unskilled" positions, their scoring is opaque and arbitrary. I don't think they even know what a sensitivity analysis is, let alone having any notion of a success correlation to measure against. They've never even discussed the arbitrary weightings they use, let alone try to justify them.
They don't even divide what scoring they do per play, so their scores as reported are inherently not even useful for comparative purposes.
They use a subjective scoring system subject to "Russian Judge" problems, and I see no evidence they have made any attempt to even normalize their scorers vs each other.
It's such a shame, really, because the put so much effort into it, and done properly it could be a unique and useful service, especially for league-wide comparisons of players in non-skill positions, who are very poorly served by pure statistical systems like the NFL's official stats or Football Outsiders' system.
So in my book, they get an A for the effort that they claim, but a D- for the execution they demonstrate. Until they put all their work behind a paywall, I would have given them a C- for the (limited) utility they provide, but that's a fail now too.
And I'm one of their biggest supporters here
