In the snow, Indy would be slaughtered wholesale. Freeney and Mathis would be taken out of the game, and the Colts D, built on speed, is going to struggle against the run game and even the passing game (because of the lack of pass rush). We all know Manning's tendencies in the snow; see 2003 Playoffs for a primer if you don't know.
If the weather is clear, the game is closer, but I don't see the Colts coming in and winning the game. I feel the same way about the Ravens. The Ravens are probably the same team they were in October, but I wouldn't say the same is true of the Pats.
Maybe it's a bit arrogant, but I think these discussions are like the tractor trailer deciding whether or not the Prius or the Aveo is the bigger threat. BB, TFB, home field, maturing and talented defense, the variable offensive sets and gameplans, etc. I'm not getting all that worried yet.