This post pretty much sums up the complaints about the Pats defense. Points per game is a valuable stat, but especially for regular season performance. And the Pats offense (lack of turnovers especially on the Pats side of the field) and special teams have greatly contributed to the Pats defensive points per game stats. The Pats being number 1 in the league in terms of opponents starting field position is a significant contributing factor in terms of defensive points allowed.
Yes, we're all spoiled, but the issue most have about this defense is not whether it is good enough to hold off the Jets or 49ers or Browns or Steelers without Ben, but whether it is good enough to win in the playoffs against a team with a good enough defense to slow down our offense. That's why defensive efficiency stats, especially defensive passing efficiency stats are as or more important to playoff success than points per game stats.