Fully admitting I havent watched the Welker hit yet, but if its as you say, then the same people crying "dirty play" are probably the same group that think Pollard's tackle on Gronkowski where he suffered the high ankle sprain was dirty. Just a guy wrapping him up, just like Swearinger made a completely legal hit, as you say.
Whenever a fan favorite gets injured its always a "dirty hit", it's just how it is.
Edit:
I just watched the Welker play, had Welker kept his feet up, he would have been fine, still a big hit, maybe the wind knocked out of him, but Welker either lost his footing or tried to go down and lowered his face into Swearinger's shoulder, it was an intentional big hit, but it was NOT an intentional hit to the head.
Welker is either falling accidentally or intentionally lowers his own head which causes the shoulder-to-helmet blow. Not Swearinger's fault in my opinion.
I still really hope Welker gets out while he still can. He's a great guy by all accounts and he needs to leave the game before it takes a permanent toll on him like what happened to Seau and others.