First, Tom Brady > Peyton Manning. I only typed that because there are still some doubters. The BB/Fox difference is too vast to even bother explaining. Nobody in the world would dare even hint at suggesting they're in the same tier, not even Fox.
But as for the differences, a lot of it has to do with Seattle's defense from 2013 to 2014. Below is a quick comparison of some key stats between the two years:
Sacks: 43 to 36
INTs: 28 to 13
FF: 24 to 19
PPG: 14.4 to 15.9
The 2013 Seahawks had lots more depth as well. As the team matured and won a championship, some guys who had been bit players got better offers elsewhere. Here's a quick overview of key contributors who left the 2013 team on defense:
Clinton McDonald (5.5 sacks)
Chris Clemons (4.5 sacks)
Red Bryant (1.5 sacks, run stuffer)
Brandon Browner (starting corner)
The injuries in the secondary were already discussed as part of the Super Bowl coverage, but don't forget about a season-ending injury to Brandon Mebane, impacting the depth even further.
Again, this isn't to make excuses. Every team deals with injuries. But it's not exactly an apples-to-apples comparison.