I think the only way he could lose his job is if Judge Berman orders the NFL to turn over the interview transcripts and notes from the Wells investigation and it exposes the entire investigation as a sham or the NFL refuses to give that over because of attorney-client privilege and the media scream bloody murder for the double standard of punishing Brady for not giving over his cell phone and then claiming privilege and/or the fact that the supposed independent investigation wasn't an independent investigation at all. Just a loss in court will not be enough for him to lose his job.
I agree, the NFL is terrified that Berman is going to order the Wells notes unsealed and that all of the conspiracy and duplicity will become public knowledge. If Berman goes that route them Gödel will be in really deep sh.t as it will be impossible to stay on as commissioner when he has zero credibility and public support.