Sure, in some ideal world moral sense you can't be held accountable for that. But that matters precisely how in this case? The only place NE is allowed to appeal a Goodell ruling to is...(drumroll)...Goodell. And people outside of the NE fan base Are. Not. Going. To. Care. About. Our. Objections. And that's why Goodell will be able to get away with it if he chooses to go that route. If anything, Goodell will get more flak if he doesn't punish NE.
As for fining the Packers, or the Bucs, or whatever, were any of their balls measured and found to be out of spec? No, they were not. So Goodell does not "have" to do a damned thing to them. So that's a non sequitur. And as part of his announcement he can also say "Blah blah blah new process blah blah and league-wide, balls will now be measured at halftime blah blah and if they are out of spec there will be punishment."
Now I do not want this to happen. I want NE to be completely exonerated. And I think there is a chance (1 in 3, perhaps) that they will be. But I also think people saying how the NFL "can't" do this or that are naive and overlooking the realpolitik in play.