My working theory is this:
BB and Brady probably don't mess with the ball inflation except if it feels low. The equipment managers likely fill to 12.5, break the balls in, Brady chooses the balls. Some may be lower by this point, but whatever unless it's noticeable they don't get anal. If a ref wants more air, they put more air in. Most refs don't really care unless it feels like it's low or high.
Irsay may have been tipped off from someone that they had a Patriots ball and it was god help us only 11 or 12 PSI. Maybe the Indy manager is anal retentive and fills every ball to exactly 13.0, or maybe he just sharpshoots other people cuz he's an asshat. Irsay gets word and decides he'll complain if they start getting their ass kicked.
So halfway through the game he tells the refs to check Patriot balls, and the ref is probably thinking "are you ****ing kidding me?" But they comply and gadzooks! Due to the weather and the lack of exacting measures some balls are less than 12.5. So the ref has a few pumps put in (probably in the cold) and they move on.
Then, facing the inquisition, the refs are like, "yeah we totally checked the pressure before the game" so now it's a Bermuda triangle mystery. But BB and Brady don't know anything, and neither does anyone else, and the refs don't want to fess up that they half assed something so the only other possibility is FOUL PLAY. But they can't find any because none exists.
I wouldn't be surprised if after calling the league office Irsay got a bit of "your making a mountain out of a molehill, but fine we'll look into it." At which point he got pissed and decided to leak the story to his reporter buddy to push the issue.