This isn't even what PFF tries to do analytically [note: was responding to someone talking about how it was quantitative analysis], and you're making the mistake they want you to make by equating numbers with true statistical analysis - which, I'll add, is very powerful even in football. Statistical analysis generally requires probabilistic statements - we can never be one hundred percent sure that X is true, but we can say that it will occur given A, B, and C with N degree of certainty. Generally, there's a great deal of statistical error - randomness - in these models when it comes to sports (or anything, really) and the job of the statistician/sabermetrician is to find what other variables can go into the model to better explain phenomena. To get this sort of analysis, you need to read research papers of the kind that are presented at the Sloan MIT Analytics conference. DVOA and DYAR and the like come closer because they use a model to generate advanced analytics, and those analytics feel reliable, but outside of QBs, they're mostly team metrics.
Pro Football Focus, on the other hand, has volunteers sitting there and watching each play of a game and grading players on every play. This is inherently valuable. I don't watch every play to see if Dan Connolly messed up, but the PFF folks do. Now, of course, they're working with incomplete information so they have to make subjective calls as to whether someone screwed up. We don't know what the rubric they're grading from looks like. Like any grading, it's subject to individual preference - an A in one class is not an A in another class, just like a +10 for one grader at PFF may not be a +10 for another grader. Tom Brady and Drew Brees could have the same exact game but the former could receive a +8 and the latter a +11 just because the Patriots grader is more stingy or critical than the Saints grader.
Moreover, the numbers aren't really scalar metrics, despite appearing as such. There's no logical way to differentiate said +11 from +8; does it mean that Brees was "3" better than Brady? 3 of what? It's really just a stand-in to say "this player performed very well" or "this player was slightly above average" or "this player was crap." This is why presenting their grades as sortable and rankable is total bunk, done to generate conversation and website clicks (Brady is #98 and Rivers is #44, for example)
However, the value is that you know that players who were awarded good grades or bad grades for a given game probably played well or played poorly because their performance was pored over by people who have a lot more free time than any of us (except maybe Brady6). PFF's volunteers are the sort of weird OCD people who spend an hour grading the performance of a right guard 16 weeks a year. Just by virtue of having their eyes on every play multiple times, they see performances that we don't just because we don't have time to rewatch games over and over. It's an alternative to a lot of the narrative-driven analysis you get at places like ESPN, where groupthink about a particular player overrides the truth. "Tony Romo isn't clutch" is the perfect example of that. You just have to triangulate PFF analysis with other sources.
That's why I like the site, but it's important to keep in mind that their grades aren't statistics (some of their advanced statistics are indeed statistics, but it's hard to differentiate them and they aren't transparent with their calculations or rubric) and that the grades should be used as a general guide to who's a good player rather than as a way to say good player X is better than good player Y because that's an exercise in futility no matter what stats you're using to justify it.
For example, JJ Watt, Lavonte David, Darrelle Revis, and Richard Sherman all graded out extremely well in PFF and they're known to be outstanding players anyways. Revis is interesting because the narrative analysis was that he had "a down year" last season, but this analysis says Devin McCourty graded very well. Then there are some guys you never really hear about who graded very well like Stephen Tulloch, Dontari Poe, Damon Harrison, and Jason Kelce. That's interesting, and it encourages me to take a closer look at those players from other sources, like their local media or fanbase who may already know that they're a diamond in the rough who you never hear about because they're not part of the narrative.
Its our responsibility as knowledgeable fans to sift through the BS and determine what is valuable and what isn't. Unfortunately, too many of mediots and some of us, are simply too lazy to make that effort. We just eat up whatever pablum the mediots feed us like a hungry 10 month old.
