I've yet to see anyone point to a part of the rule that should have applied that would have caused the flag to be picked up.
In the replay, it's clear that he runs out there and engages Harris at least 3 yards past the line, which should be illegal as far as I can tell. Maybe, without benefit of replay, one of the refs thought that they were engaged from the LOS out, or that the pass was away before he got out there, or... To me, it seems pretty clear the the refs screwed up, but I haven't heard anything about what the refs thought they saw to cause them to pick it up.
As usual, a missed/bad call is only worth discussing if it should have been called against the Pats. If it works out in the opposition's favor, it's like it never even happened.