For me, it really hasn't been about Brady, it's just been about poor football, especially on offense. The Cam / Covid era was totally forgettable. Passes did just hit the ground for no reason anyone could explain. The MP as OC era was totally forgettable, enough said. On top of these "highlights" the things that we used to like about the Patriots, like playing good solid football with few penalties, seem to no longer be a given. There just hasn't been a lot to like about the last few seasons, therefore I watch less.
In the current era, if we're playing a night game, it's pretty easy to say to myself I don't really need to watch this, it'll be on the DVR tomorrow if I want to watch it. Then, the next day, chances are good I've already heard all about the game and seen most of the highlights so I skip watching it or maybe watch some of the action but not the whole game.