I already posted the most obvious explanation.
First, the 3rd party video I posted shows that a Wilson football drops from 13 PSI to 9 PSI in just 30 minutes when brought outside from a guy's basement to a chilly outdoors, as measured with digital and manual temperature readers and air pressure gauges and pump. In theory the air pressure may drop to 10.5 PSI, but reality shows it can go far lower (this is even before you account that a wet ball would be even colder inside than a dry ball at the same temperature.)
More than malice, the clear and likely explanation is that the visiting team, the Colts, arrived with footballs in the luggage compartment of their bus and left them there until the referees asked to see them.
The luggage area is probably not heated. Also, luggage is below the passenger area, so there would be little heat transfer into the luggage part of the bus because heat travels upwards toward the roof. Finally, the bus was likely parked with the engine and heat off after the team arrived.
The Colts balls were likely left on the parked bus until they were needed by the referees for obvious reasons -- why bring them into the host's stadium. It was also raining heavily and there was a chance the rain could stop, so why get them wet --twice -- right before the game. Why carry 24 valuable footballs (12 of theirs, plus 12 optional the visiting team can bring) back and forth in heavy rain?
On the other hand, I doubt Bill or Tom keeps game footballs in the trunks of their cars or an outdoor shack. The host team (Patriots) obviously stores the balls inside and probably inflated them inside.
They inflated them to 12.5 PSI, because the NFL allows all quarterbacks to choose the pressure they like when they give the balls to the referees. Either the referees measured the Patriots footballs indoors (especially with heavy rain) or they were brought out to the referees outside shortly before they were measured, so they still had the right air pressure. In fact, with heavy rain or snow, and hours to kill before the game, I bet the referees will check the visiting team balls outside and the host team inside -- they don't need to do both simultaneously.
A drop of 4 PSI or less is well within the theory and 3rd party tests of what's expected. Additionally, the Patriots are not allowed to tamper with the balls (by heating or inflating) after the referees approve the starting air pressure. Then we would have InflateGate. And we already know the referees don't care about the air pressure between plays so I doubt they would hold up a play to add air to the ball at a quarterback's request (maybe in the future, but not in the past.)
Finally, at half-time, the referees obviously pumped more air on the field, not in a special room indoors, and it was cold outdoor air so the balls maintained pressure throughout. This proves the balls were not leaking air, and not faulty in any way. Finally, the balls lost pressure uniformly. If you wanted to cheat, you would inflate the balls at different levels and mark them, so you could pick and choose based on weather conditions during a particular hour.