Passing yards isn't a terribly useful stat, so lets look at where NE ranks in more meaningful categories.
Y/A - 28th (8.7)
Comp % - 25th (72.1)
PR - 16th (93.6)
VOA - 9th (-2.4%)
So, if the pass gets off, it has a high likelihood of being successful, but the Patriots have done a good job of countering those success with big plays.
Interestingly, you would think that, since VOA is that positive about NE's pass defense, they would have a favorable overall rating. Not so. They have the Pats 21st and comfortably below average, largely due to terrible run defense (31st, 16.6%).