Not necessarily. Points allowed is not the" only stat that matters" because you have to look at the teams and offenses you're playing. The Patriots defense has allowed huge chunks of yardage to mediocre quarterbacks, mediocre offenses, and in multiple cases BACKUP quarterbacks. Every time they mention an opposing quarterback's stats I swear they're 13/15 or completed their last 11 passes straight or something. In more than a few cases the Pats defense has been bailed out by unforced offensive errors, penalties, or missed field goals. Yes, those mediocre offenses weren't able to take advantage, but the Pats D can't depend on that in the playoffs.
Look at what's happening in this Steelers game. Below average defending against a backup quarterback in Landry Jones who's barely played (if at all?). It's too easy to gain chunks of yardage on this supposedly good defense. Against a good offensive team it's really concerning because they'll take advantage of all the opportunities afforded by the ease with which they can move the ball.
