Bay isn't as bad as Dunn, but he is BAD. (nobody is as bad as Dunn).
He makes terrible jumps, is slow to the ball, and doesn't throw well. Drew, on the other hand, is very good in the outfield.
Ellsbury, who you mentioned below, is kind of a question mark. You have to consider that hes being compared against much better players (other centerfielders). He's good, but he tends to read the ball poorly (but often makes up for it with his speed), and is a mess near the wall.
As to the poster who mentioned Teixera, hes similar to Jeter: hes very steady handed, and has a fantastic glove. He just has no range.
Cabrera is the same, his range has completely eroded the last couple years.
Also, the Gold Glove is voted on by a bunch of sportswriters. Its not really proof of anything.