For the latter point, Vinatieri wasn't particularly great at kickoffs. A friend of the family is a scout in the Pats organization, and he said that one of the factors in letting Vinatieri walk was exactly that. Not a decisive point or anything, I wouldn't think, but it did factor in.
For the first point, I'd just say that the NT that's dominating the LOS, or the LT who's keeping the heat off of Brady's blind side, or the pass rusher who's bringing the heat and keeping the opposing offense from getting in rhythm... I don't see how you could argue that a kicker has a stronger impact on the game than those factors. A good player in these positions will directly affect the outcome of the game, to use your words, on at least as many plays as the kicker did. And for the rest of the plays where he's on the field... well, he's out on the field contributing, which the kicker, through no fault of his own, isn't.
It's fine, I expect my opinion is in the minority here, and it's a difficult one because I'm not sure how to argue it factually outside of snap count. It's more a principle thing, which is why I don't intend to argue it too hard and admit I may be totally off base on this one.