I can certainly see some griping about the defense, as there are problems (I think Metaphors is correct that they are not forcing QB's off their first read), but the overwhelming "defense is crap" sentiment is silly and not based in fact.
CHFF has a great defensive stat called bendability. It's a quality stat that has been proven to have a direct correlation to wins/loses. It's basically yds given up per point, so it blends yardage allowed and points allowed. Last year, for the whole year we were at 18.73, and Green Bay (eventual SB Champs) were tops overall at 20.61. Pittsburgh (who lost the SB) was second at 19.06. We were 4th behind Baltimore. The year before our rating was 17.98 (which was again 4th in the league).
After 3 games this year, we are at 17.82, which is good for about 6th-7th in the NFL right now, ahead of teams like Buffalo, Pittsburgh and the Jets. We're giving up lots of yards, but not nearly as many points. At this point last year, we were at 13.60 and were allowing 27.66 ppg (more then we are now). We were at the same record to (2-1). We improved significantly over the course of the season and there is no reason to believe we won't again. Statistically, this defense is playing BETTER then we were last year at this point by this measure, which again correlates directly to wins/losses.
Lots of room to improve, but it's not NEARLY as bleak as people might be inclined to think.
If you absolutely MUST place blame for this loss, look to the other side of the ball. Yeah...... I'm serious. We gave them 4 extra possessions on offense and they made the most of it. Bills scored 24 points off of Int's. Sure it was against this defense, but lets be honest, they should have NEVER been in the position in the first place. We had a couple Int's of our own, and we managed a meager 7 points of them. The turnover/point differential was 24-7...... You ain't EVER gonna win games like that. Take out those giveaways and we dominated this Buffalo team on both sides of the ball 24-10. At the game, it really did feel like we dominated them outside of all those crappy Int's and the ensuing drives afterword.
You HAVE TO EXPECT a defense to take advantage of giveaways to the tune of 3 or 7 points. Sunday was NO EXCEPTION. Buffalo scored on all 4 (3 TD, 1 FG) and we scored only on one (1 TD). In fact, in a game that we moved the ball at will, one of our only 3 and outs came after the second Int. That's uncharacteristic. Brady almost ALWAYS makes em' pay. It's a signature thing for him.
You just can't do that kind of stuff. Sunday was a rare day for the offense in that 1) they were careless with the rock, and 2) they did not take advantage of Buffalo's mistakes. Had they done either, we would have won this game. Plain and simple, the offense gave this game away.
There really isn't much of an argument the other way beyond "they gave up alot of yards" and a few easy TD's (on possessions the Bills SHOULD NEVER HAVE HAD and in situations you can reasonably expect to get scored on).