So here's what I'm trying to do with the "old timer" problem that comes up on a lot of these lists.
There are three issues with the older players:
1. The pre-1950 data is incomplete, and the further back you go, the less reliable it gets.
2. The data itself has some problems; some of the QB stats like win % is complicated because the QB wasn't always the starter, might have been flexing in different positions, etc. Also, while passer rating is being used as a general guage, there are some really big outliers with huge inflated passer ratings despite completing less than 50% of passes; Luckman is a good example. The quarterback's role in the offense changes over time, but it's a pretty extreme contrast when you get into pre-1950.
3. There are also overall league infrastructure problems. Borderline amateur teams rotating in-and-out; rival leagues watering down the competition, etc. These problems continued to exist until the merger; that's why even 1950s and AFL/NFL accomplishments are harder to give full credit for; in the 1960s, these two leagues were almost equals, so you're really playing the equivalent of the AFC and NFC for competition.
I think that players should keep all of their accomplishments. Championships, awards...I don't like the idea of watering down those wins. A lot of these players were trailblazers for the league and pushed the standard higher at the QB position.
It's the performance scores that are the problem, as the wild fluctuations, lack of confidence in the numbers, etc. can cause problems everywhere by overadjusting, and of course, the overall fairness. The other big problem here is that, with era-adjusting the passer rating, even the league average passer rating suffers from the same issues as the individual one I'm measuring. So, to simplify, there are problems with the two major stats I'm using to create a performance score, which is a huge part of the player's ranking:
-Passer Rating (and Peak Season Passer Rating)
-Winning Percentage
What I think might be a good compromise - and what I've done in the last rankings - is create a sliding adjustment called a "Confidence Index." The way it works is like this: In 1967, the player would receive 100% of his performance score. In 1966, the player would receive 98% or so; in 1965, 97% or so. This goes down so that when we get to the 1930s, a fairly big adjustment is made. The player's confidence rating is on a year-by-year...so if you played two years, and the confidence rating was 60% and 58%, you get the average: 59%.
One point that's really important here: what this accomplishes is more "regression to the mean" than penalty. Since these ratings are based on relative era-adjustments, 0 is average. So a player with a -7 score, might only get a -4 after this adjustment. One the peak index, Sid Luckman went from having the greatest peak score ever to a very good one, still among the best ever. Otto Graham's performance stats are still really, really good, but he's no longer a ridiculous outlier who breaks the system.
Here's the chart which shows the current configurations, how much adjustment is made for each player relative to their raw scores (don't focus on the raw score because it's combining longevity, efficiency, and also some formla stuff I'm experimenting). I'd keep in mind that, even using this confidence adjustment, it still didn't kick out Baugh or Luckman from the top 10 but certainly brought some of their inflated scores down to make them less untouchable. The big plus is it can correct some outliers by bringing guys towards the mean, and it also helped out some players (they aren't shown here because they were so far down) to not take such a huge hit for a really low passer rating in the 1920s when there are only records of them throwing like 40 passes. And again, that's part of the confidence issue too, is the confidence in how much we're comparing quarterback skills to each other; so when those skills are very much in question, smoothing out the performance index and deferring to championships and awards is ideal, I think.
View attachment 30930
