I think the biggest problem with PFF is that they try to quantify the unquantifiable, and that in itself wouldn't be as bad if people didn't reference that false logic as anything more than a guess.
They don't even really quantify the unquantifiable. Their numbers are just tricks that conjure up a mathematical mystique to disguise a qualitative analysis.
The best analogy to PFF is this: a 90 on an English essay isn't quantifying anything, it's just an arbitrary numeric representation of the grader's opinion about your paper, which can be affected by any number of factors including whether the grader agrees with your thesis, whether they're having a bad day, whether they like or don't like you, and so on.
The difference between a 90 and an 85 on an English paper is totally arbitrary, versus, say, the difference between a 90 and an 85 on a 20 question multiple choice math test where each question is worth exactly 5 points. Then the difference between a 90 and 85 represents one correct answer.
Once you recognize what PFF's doing here, you realize it's no different than anyone else doing a "performance review" of a player, similar to Reiss's 3 up, 3 down. The
positive side of PFF is that they watch every player and grade every single player, which is more than most analytical websites. If they stopped there and just provided a qualitative critique of each player, I think they'd be much more valued.
The problem is they go the extra step and try to turn that qualitative critique into numbers to make believe they're doing "metrics" and that's where you get a lot of the skepticism. There doesn't appear to be any rationalized system of grading, and the difference between a +3 and a +4, for example, is totally opaque. And I expect the reason for that is that it's totally arbitrary, like the aforementioned English test.
If you can turn their grades back into vague representations of a player's performance (that is, a +4 represents a "good performance" or whatever) then PFF is fine. Of course, this still loses a lot - they graded Browner on the whole fairly poorly, but anyone who watched Browner knows there's a reason for this: against some receivers he's a total shutdown corner, and against others he's toast. You lose that nuance when you try to turn a qualitative performance review into an arbitrary grade.