I broke these up into several categories, though there is enough gray area that many of these uniform numbers can easily be placed in a different section. It turns out there are a lot more difficult/debatable choices for 'best jersey number ever' than I thought there would be, and a few that are virtually impossible to determine.
------------
The Obvious Choices:
#4 -- Adam Vinatieri -- (next closest, Jason Staurovsky)
#12 -- Tom Brady -- (next, Matt Cavanaugh)
#14 -- Steve Grogan -- (Tom Yewcic, Vinny Testaverde, Zoltan Mesko...)
#15 -- Babe Parilli -- (Chris Hogan, Marc Wilson, Kelley Washington, Ryan Mallett)
#20 -- Gino Cappelletti -- deserves to be in HoF; nobody else has ever worn #20
#24 -- Ty Law -- (**** Felt, Darrelle Revis, Kyle Arrington)
#26 -- Raymond Clayborn -- (Eugene Wilson)
#35 -- Jim Nance -- (Patrick Pass, Jonas Gray)
#36 -- Lawyer Milloy -- (James Sanders)
#39 -- Sam Cunningham -- (Danny Woodhead Laurence Maroney, B Browner)
#51 -- Jerod Mayo -- (Eugene Lockhart, Bob Golic...)
#55 -- Willie McGinest -- (even though Don Blackmon was very good)
#57 -- Steve Nelson -- (nobody else has worn #57 in thirty years)
#65 -- Houston Antwine -- (Damien Woody is a distant second)
#73 -- John Hannah
#75 -- Vince Wilfork
#78 -- Bruce Armstrong
#79 -- Jim Lee Hunt
#83 -- Wes Welker -- (honorable mention to Deion Branch [1st stint], 2002-05)
#86 -- Stanley Morgan -- (also worn by David Patten)
#89 -- Bob Dee -- one of only two numbers to have only been worn by one player
#93 -- Richard Seymour
The Easy Choices:
#1 -- John Smith -- (over his successor, Tony Franklin)
#3, Stephen Gostkowski -- (even though Rich Camarillo is the Pats best all-time P)
#16 -- Jim Plunkett -- (Matt Cassell only started one season)
#31 -- Fred Marion -- (Brandon Meriweather, Aqib Talib)
#38 -- Roland James -- (Brandon Bolden)
#48 -- Tim Fox -- (only other starter was Don Webb, before he switched to #42)
#56 -- Andre Tippett -- (honorable mention to Jon Morris and Rod Shoate)
#58 -- Pete Brock -- (ST Matt Chatham may be the second best #58...)
#70 -- Logan Mankins -- (Leon Gray was very good, but had only six years w/Pats)
#87 -- Rob Gronkowski -- (over Ben Coates, David Givens)
Numbers Worth Mentioning a Second or Third Player:
#18 -- Randy Vataha - (Matt Slater and Donte' Stallworth)
#32 -- Devin McCourty - (Andy Johnson, Antowain Smith, Craig James)
#33 -- Kevin Faulk - (over Tony Collins)
#42 -- Ronnie Lippett - (Don Webb, Mack Herron, BJGE)
#44 -- Don Calhoun - (over John Stephens)
#59 -- Vincent Brown - (over Rosevelt Colvin)
#60 -- Len St. Jean - (over Garin Veris)
Difficult Choices:
#11 -- Drew Bledsoe > Julian Edelman
#22 -- Asante Samuel > Ron Burton and Stevan Ridley
#28 -- Curtis Martin > Corey Dillon
#37 -- Rodney Harrison > Maurice Hurst
#40 -- Mike Haynes > Larry Garron
#50 -- Mike Vrabel > Rob Ninkovich (and Larry McGrew and Sam Hunt)
#52 -- Ted Johnson > Johnny Rembert
#54 -- Tedy Bruschi > Dont'a Hightower (though that may eventually change)
#61 -- Sam Adams > Stephen Neal and Ron Wooten
#67 -- Bill Lenkaitis > Dan Koppen (very close call here)
#72 -- Matt Light > Larry Eisenhauer
#76 -- Sebastian Vollmer > Charlie Long and Brian Holloway (?)
#77 -- Nate Solder > Tom Neville, (and Nick Kaczur, Pat Harlow, Kenneth Sims)
#81 -- Jim Colclough > Russ Francis > Randy Moss (???)
#82 -- Daniel Graham > Jim Whalen > Vincent Brisby (??)
#84 -- Darryl Stingley > Shawn Jefferson, Art Graham, Ben Watson, Deion Branch (2nd stint, 2010-12) (?)
#85 -- Julius Adams > Nick Buoniconti (can only consider time spent w/Patriots)
More Close Calls Worth Debating:
#25 -- Rick Sanford > Ross O'Hanley
#27 -- Ellis Hobbs > Ron Bolton
#29 -- Harold Jackson > LeGarrette Blount
#30 -- Mosi Tatupu > Carl Garrett
#53 -- Tom Addison > Chris Slade (and Larry Izzo and Clayton Weisuhn)
#62 -- Ryan Wendell > Dwight Wheeler
#64 -- Richard Bishop > Dave Wohlabaugh
#68 -- Max Lane > Tom Ashworth
#71 -- Ray Hamilton > Todd Rucci
#80-- Troy Brown > Irving Fryar
#90 -- Chad Eaton > Toby Williams
#91 -- Jamie Collins > Bobby Hamilton (?)
#95 -- Chandler Jones > Roman Phifer
#98 -- Anthony Pleasant > Keith Traylor, Dennis Owens, Brandon Mitchell (?)
Default Choices; not much competition for these uniform numbers:
#6 -- Ryan Allen (over Chris Hanson)
#10 -- Jabar Gaffney (maybe Jimmy Garoppolo can move #10 into another category in the future)
#19 -- Brandon LaFell (Brandon Tate, Tom Tupa; Malcolm Mitchell could take over as the best #19 by the time he signs a second contract)
#21 -- Malcolm Butler (Ricky Reynolds and 19 others have worn #21, but Butler is already far and away the best of the group)
#23 -- Ron Hall (over Horace Ivory)
#34 -- Chuck Shonta (over Prentice McCray, Sammy Morris, Ron Sellers)
#45 -- Otis Smith (over 15 JAGs)
#47 -- Robert Edwards; does #47 fall into the 'jinxed' category, like #17?
#69 -- Shaq Mason > Eugene Chung
#74 -- Shelby Jordan > Kyle Love
#88 -- Terry Glenn > Martellus Bennett
#92 -- Ted Washington > Ray Agnew, Ferric Collins
#94 -- Ty Warren > Chris Jones
#96 -- Brent Williams > Adalius Thomas, Mike Jones
#97 -- Jarvis Green > Alan Branch, Mark Wheeler
#99. -- Mike Wright > 16 JAGs
Numbers in dire need of a good player:
#2 -- Doug Flutie (over Mike Patrick)
#5 -- Shayne Graham (over Pat O'Neill)
#7 -- Hugh Millen (Chalie Gogolak, Michael Bishop)
#8 -- Josh Miller (Charlie Baumann)
#9 -- Four guys who combined to play a total of 13 games with the Pats. Bryan Wagner by default over two horrible kickers, David Posey (11/22) and Scott 'Missin' Sisson (14/26).
#13 -- RC Gamble? Tommy Hodson? Not Joey Galloway...
#17 -- Aaron Dobson or Mike Taliaferro? 13 players have done nothing with #17
#41 -- Larry Carwell, Leroy Mitchell; 18 players including Ray Ventrone, Cyrus Jones.
#43 -- Nate Ebner ... not good when a special team specialist is the best #43 ever; others include Vagas Ferguson, Ernest Gibson.
#46 -- James Develin (over Marv Cook and 11 others)
#49 -- 2-year LS Joe Cardona, or 1-year starting FS Ralph Anderson?
#66 -- LS Lonie Paxton over 2-year starting G Paul Fairchild