Well, the Packers are the #1 seed in the NFC, and they have the 31st ranked defense in the league (by yards allowed, which is also how the Pats are ranked #32).
So the top teams in each conference have gotten there despite the two lowest ranked defenses in football.
Yards allowed is a weak statistic upon which to base a judgement of "best or worst" defense.
Both of these teams pummel their opposition and get big leads. The defense shifts to keeping everything in front of you and the opposing offenses abandon the time-consuming running game and start flinging it all over the joint. Opponents rack up yardage. So what?
Here are a few stats about the defenses of the Packers and Patriots to think about:
Opponent Scoring (a pretty good measure): The Patriots and Packers are tied at 14th in the NFL with giving up 297 points - ten points better than the NFL average of 306 points through 14 weeks. ( The 49ers are a ridiculous 185 - the only team under 200 points allowed.)
Opponent Rushing TDs: The Packers and Patriots are average in this category. The NFL average is 11 TDs allowed on the ground. the pats have given up 12 and the Packers have given up 10 - both less than one a game. these teams are tough in the Red Zone.
Field Goals against: The Pats and Packers are tied with the 49ers at #4 in the NFL in Field Goals allowed. Don't know what to make of this really, but interesting cluster of the top three teams in the NFL.
Interceptions: The Packers are #1 in the league with 27 interceptions and the Patriots are #4 in the NFL with 18 interceptions through 14 games. These two teams make the opposition throw the ball trying to come from behind and get takeaways.
Defensive Penalties: The Packers are #2 and the Patriots are #9 in 1st downs allowed on defensive penalties. They are making the opposition earn what they get and not handing the other team opportunities to keep drives alive.
The other statistical area where both teams excel is special teams defense. Both are top 10 in punt and kickoff return yardage allowed. The Patriots have not given up a TD in the kicking game and the Packers just one.
In lots of ways, these are two average defenses paired with great offenses.