People are often confused on this point so I'll mention it.
Fair catch interference does not apply only to punts where the returner signals for a fair catch. The returner is not allowed to be hit at any time before receiving the ball, unless the contact is deemed unavoidable. The fair catch signal simply protects the returner from being hit AFTER he makes the catch. The returner has the right to catch the ball unimpeded and it is in fact a penalty (illegal touching) for the kicking team to touch the ball before a returner, just as it is for them to disrupt the returner from making that first touch (fair catch interference).
Therefore, whether the gunner saw Edelman's signal is irrelevant when considering whether the hit was illegal or dirty. The gunner knows he can't hit a returner until the returner has touched the ball.
The extenuating circumstances that may have entered into the ref's decision here were that (a) Edelman was on the move, and (b) the gunner was being blocked. If the returner makes a sudden move to catch the ball sometimes the gunner is allowed to make contact (contact is unavoidable; the gunner does not have eyes in the back of his head to track the ball) and obviously if the gunner is shoved directly into the returner (such as in an attempt to draw a foul) contact is again unavoidable.
Given that both Edelman was moving rapidly and the gunner was being blocked up to the moment of contact, and the ref was watching it all in real time and not slo-mo from the best camera angle available, perhaps the ref though the contact was unavoidable. Personally I feel the gunner was in complete control of himself and made no attempt whatsoever to avoid illegal contact but that would be why the penalty wasn't called on the field.
I wouldn't be surprised if the gunner was fined.