Any Statisticians out there? 538 weighs in on Spygate.

[Joker]

Fact...Goodell worked for the Jets FIRST then was hired by the league

Fact...his work history was ALTERED after this was published in the media...the media then used the "Goodell worked for the league...then went to work for the Jets as an intern and then went back to work for the league" fabrication in ALL subsequent reports on his work history. IT"S ALL A BLATANT LIE AND SHAM.

Fact....Mangini and his Jets were caught using TWO cameras field level against the Pats and asked by Belichick to remove them. Goodell DID NOTHING!!!! FACT!!!!

The Thursday before the Spygate story broke the first game of the season, Mangini signed into NFL offices, where the OFFICE OF COMMISSIONER GOODELL IS LOCATED, at 3:30 P.M. WHY? What was discussed. What was the purpose of this visit. NO MEDIA WILL ASK WHY. I HAVE copies of the sign-in dated and signed. FACT.

Goodell essentially drops a NUKE on the Pats and their entire organization as well as US, the NE Patriot fanbase, and subsequently sits there like a deaf ,dumb, blind ,submongoloidal monkey as the Patriots are vilified, excoriated , libeled and slandered AT WILL by every cretin with access to a microphone. Lie after lie after unbelievably ridiculous lie spewed by dysfunctional morons like Joey Porter and Tomlinson and that fakeazzed holy hypocrite Warner, aided and abetted by OUR own media scumbags, Tomase and Gasbaggarotti as well as the repugnant Easterbrooks of the national media.

Meanwhile, what's going on in Jetland?? Toe degeneracy and open swinging on the internet, blatant streams of public profanities spewed by a loudmouth fat slob spoiled brat kid of the biggest jerk assistant coach in history...ice cream parties published nationwide that smear the NFL reputation, organized sideline tripping in game that was practiced and known by their entire staff, blamed on a little monkey in the middle Sal Alosi...on and on year after year...essentially NO RESPONSE OF ANY CONSEQUENCE by Goodell.

Now look at what we have...new "rules" backroomed and backdoored in for this season, right after the little Forehead Fauntleroy gets eviscerated in the Super Bowl. You've all seen it so far. It is PREPOSTEROUS. RIDICULOUS....and Goodell's handpicked overseer and implementer of these new idiocies??? Sharing hookers and strippers with a team owner's son IN OPEN PUBLIC.

But yeah, fine the Pats a million bucks, take their first round pick, alienate the fans of New England, make their children TARGETS of schoolyard bullies as they get harrassed as "cheaters!" and "scumbags". Yeah, by the way, where IS good ol' Roger the rotten little rich kid moron scumbag?? Not up here talking to any of US....NOT EVER. He's got to go. I don't care what it takes. The Feds should be looking at this cretin and crawling up his azz with investigators.I personally will never rest until I see this jerk get the justice he deserves.

 

[BradyFTW!]

I'm no statistician, but I did minor in it so I have some baseline knowledge. Disappointing article, because the only conclusion that a worthwhile statistician can make from the postseason is that the sample size is too small to draw any meaningful conclusions. Full stop.

And sure, the article paid lip service to that fact... which was followed by four paragraphs of pure garbage. The fact that this whole paragraph:

"As is usually the case when dealing with real-world data, the answer isn’t totally conclusive. From a holistic viewpoint, using all repeat-opponent games in the regular season and playoffs, there hasn’t been any difference — significant or otherwise — in the Patriots’ offensive performance since the league mandated the team stop taping opposing play calls. Looking at the playoffs alone yields a more nebulous picture but also introduces methodological questions about the aptness of conventional significance testing. And we must always keep in mind that splits happen if we look for them hard enough."

can be written without mentioning how small the playoff sample size is and the effect of injuries, etc. - all just points to crap, clickbait journalism. My Regression Modeling 101 professor would have failed me if I'd turned in a model that even attempted to draw conclusions from 12 data points without explicitly calling out the sample size issues and acknowledging outliers.

Also, let's pretend for the sake of argument that you're an analyst and you're determined to draw a conclusion from the playoff games sample. Anyone worth a damn would first check the playoff results over time to identify if there was a trend, if it shifted, and, if so, when it shifted. If you were going to do that, you'd pretty clearly see that, if any shift actually happened, it happened between 2004 and 2005. As a result, placing it at 2007 is probably arbitrary and will just catch the tail end of a shift that's already happened, thus misrepresenting the data. That's not to say that you can't do it, but there are some questions that you need to answer. And if there's something that happened between 2004 and 2005--like, for example, a veteran and coach exodus, a captain having a stroke, and an officiating point of emphasis that counters how the Pats defend--then it doesn't take Albert ****ing Einstein to realize that that's your answer. There's the cause for whatever shift you've observed.

Lastly, using Vegas spreads is a pretty major red flag in its own right. Anyone on either side of the Spygate debate can blow up the entire premise by claiming that the effects of Spygate (if there are any) are both directly and indirectly factored into Vegas lines. With that one, trivial sentence--something that even a casual football fan should easily deduce--this entire article becomes an exercise in futility.

Which sucks, since the whole point of FiveThirtyEight is supposed to be that it's above that. They should leave lying with statistics to the clown over at Unskewed Polls and all the other right wing counter-Silvers who spent a year plus inventing reasons why Romney would get elected in 2012.

 

[primetime]

Which sucks, since the whole point of FiveThirtyEight is supposed to be that it's above that. They should leave lying with statistics to the clown over at Unskewed Polls and all the other right wing counter-Silvers who spent a year plus inventing reasons why Romney would get elected in 2012.

So-called "data journalism" is largely clickbait. Vox seems to be doing a better job as the go-to emergent site because they don't try to explain everything in some chart or freshman level quantitative analysis and have real subject matter experts plus Klein and Yglesias, both of whom are highly readable and obviously incredibly intelligent though can be obnoxious in their own ways in shaping every story to fit their narratives. But I digress.

Your post is all correct but it even goes beyond the 12 data points thing though because games aren't really a useful level of analysis in football at all anyways. If you wanted a quantitative study on this, it's possible but would be extraordinarily complicated and definitely not worth it for a 1,000 word 538 article. You'd have to look at success rates on individual plays with a cut-off from Spygate, controlled for opponents and about a dozen other things.

Even that's pretty fraught though. Controlling for opponent can get odd, for one. Second, there's still a ton of randomness. And, last but not at all least, the causal mechanism is really not particularly strong with the argument that the Patriots were gaining an advantage no other team had that they suddenly lost at the end of that Jets game somehow even though literally every coach asked about it has said they didn't, so we can't even be sure the quant study's results are telling us anything useful other than digging for noise.

This is one of those things where a quantitative study, while possible, would be hugely complicated and really wouldn't tell you anything you couldn't get from common sense, interviews with other coaches, and eyeballing the Patriots' success as a team anyways which are all much more convenient.

 

[PATS16N0]

Fact...Goodell worked for the Jets FIRST then was hired by the league

Fact...his work history was ALTERED after this was published in the media...the media then used the "Goodell worked for the league...then went to work for the Jets as an intern and then went back to work for the league" fabrication in ALL subsequent reports on his work history. IT"S ALL A BLATANT LIE AND SHAM.

Fact....Mangini and his Jets were caught using TWO cameras field level against the Pats and asked by Belichick to remove them. Goodell DID NOTHING!!!! FACT!!!!

The Thursday before the Spygate story broke the first game of the season, Mangini signed into NFL offices, where the OFFICE OF COMMISSIONER GOODELL IS LOCATED, at 3:30 P.M. WHY? What was discussed. What was the purpose of this visit. NO MEDIA WILL ASK WHY. I HAVE copies of the sign-in dated and signed. FACT.

Goodell essentially drops a NUKE on the Pats and their entire organization as well as US, the NE Patriot fanbase, and subsequently sits there like a deaf ,dumb, blind ,submongoloidal monkey as the Patriots are vilified, excoriated , libeled and slandered AT WILL by every cretin with access to a microphone. Lie after lie after unbelievably ridiculous lie spewed by dysfunctional morons like Joey Porter and Tomlinson and that fakeazzed holy hypocrite Warner, aided and abetted by OUR own media scumbags, Tomase and Gasbaggarotti as well as the repugnant Easterbrooks of the national media.

Meanwhile, what's going on in Jetland?? Toe degeneracy and open swinging on the internet, blatant streams of public profanities spewed by a loudmouth fat slob spoiled brat kid of the biggest jerk assistant coach in history...ice cream parties published nationwide that smear the NFL reputation, organized sideline tripping in game that was practiced and known by their entire staff, blamed on a little monkey in the middle Sal Alosi...on and on year after year...essentially NO RESPONSE OF ANY CONSEQUENCE by Goodell.

Now look at what we have...new "rules" backroomed and backdoored in for this season, right after the little Forehead Fauntleroy gets eviscerated in the Super Bowl. You've all seen it so far. It is PREPOSTEROUS. RIDICULOUS....and Goodell's handpicked overseer and implementer of these new idiocies??? Sharing hookers and strippers with a team owner's son IN OPEN PUBLIC.

But yeah, fine the Pats a million bucks, take their first round pick, alienate the fans of New England, make their children TARGETS of schoolyard bullies as they get harrassed as "cheaters!" and "scumbags". Yeah, by the way, where IS good ol' Roger the rotten little rich kid moron scumbag?? Not up here talking to any of US....NOT EVER. He's got to go. I don't care what it takes. The Feds should be looking at this cretin and crawling up his azz with investigators.I personally will never rest until I see this jerk get the justice he deserves.

 

[Joker]

how the hell does anyone think they can quantify heart, grit, determination and courage/coolness under pressure?

this offseason is a perfect example...day after agonizing day, reams and reams of charts drawn from stat sites listing players numbers...every conceivable stat that exists. How many hundreds of posts on Michael Buchanan's "upside" citing these libraries of stats have inundated our board the past four months?...and yet, Thursday's exhibition showed that yeah, he's really cut, yeah he's really ripped, yeah, he's really athletic, yeah he really LOOKS like a stud NFL player...only one problem....he is a LOUSY excuse for an NFL level depth player, nevermind a starter on ANY defensive squad in the league.

All the stats in the world still cannot discern a distinction between a Vernon Gholston and a Zach Thomas...other than thousands of squawking stat geeks screaming for the no brainer Gholston and scoffing at Zach Thomas's ability to even MAKE any team.

Football first and foremost is played by FOOTBALL players...not combine warriors.

 

[PatsFanSince74]

I'm a statistician (demographer) irl

Like most 538 articles this is complete crap. First off, the playoff record is not subject to random chance for a lot of reasons both substantive and methodological.

Broadly speaking, the issue with using a football game "win" as the unit of analysis is that a win is highly random and really the result of tens or hundreds of random or near-random events. It's why judging quarterbacks on wins or Super Bowls is a really dumb way to judge them. A win can turn on a single completely random fumble (fumbles have been shown to be completely random in who recovers them). Games aren't the level of analysis for football - individual plays are. And even those are incredibly random compared to a basketball possession or baseball push, which makes statistical analysis of football highly susceptible to vagaries.

Points are also not a good way to analyze football. Points are awarded in units of 2 (rarely), 3 (often), 6 (rarely), 7 (often), or 8 (rarely). They're not regular. It's why I object highly to the "net expected points" metric that has become somewhat en vogue - football points are an ordinal and not continuous measure, compared to baseball runs for instance.

Moreover, using Vegas spreads as a way to analyze anything at all is incredibly stupid. Vegas lines don't move based on what they think the score will be or anything. They move on where the money is going in order to get a profit. The Cowboys, Patriots, Giants, and other popular teams are more likely to have money move their way than other teams, for instance. People will bet on the Patriots to win even when that isn't a sensible bet because Patriots fans have money and will bet and are confident in those bets.

I hate Neil Paine. He likes to use freshman level econometric analysis as a way to show he's a scientist or something, using things like p-values and t-tests - incredibly simplistic in the world of statistics but arcane sounding to most non-statisticians - to make it seem like he's a real analyst.

This research has a small sample size, a cherry-picked dataset (which he admits, at least), and the methodological and substantive basis of the study is underwhelming at best. If you want, print this out and use it as toilet paper. That's all it's good for - and I'll say that not as a Patriots fan, but as a statistician.

While the bottom line is that people will "stand where they sit" when it comes to Spygate and no one will change their minds one way or the other as a result of the 538 analysis, that's a cogent rebuttal. Unlike you, I'm not a professional statistician, but I use statistics all the time in my work and have managed my way around a Distribution or two.

The analysis that the author tries to do is fundamentally meaningless given the sample size, if nothing else. Your observation on how points are scored is especially to the point (as another example, predicting the outcomes of the US General Election for President completely misses the point if the analysis does not take into account the fact that electoral votes are amassed in an assortment of chunks). If the author had a universe of several thousand games, his analysis, properly adjusted, might be useful to draw conclusions but, by definition, he does not have that kind of sample of Patriots games.

All I would add to your fine analysis is that the "spread," beyond your other observations, does not take into account the number of points predicted for a game, which is captured by the OU line. So, being four points off the spread in a 59 point game is materially different than being four points off the spread in a 39 point game.

My own view of Spygate is what it's been for years, viz., Bob Kraft was right: Belichick was a "schmuck" for ignoring the league memo to do something that had been "legal" in the past but was forbidden at the time of that particular taping and that gained the Patriots no advantage on the field and caused the franchise serious brand damage (outside New England and Patriots nation) off the field. Yes, the penalties were excessive and the cowardly product of a rookie Commissioner influenced by a media frenzy that the Patriots for all intents and purposes did nothing to stop (whether or not they could have stopped it is another subject), but that's all just la-de-da.

 

[primetime]

My own view of Spygate is what it's been for years, viz., Bob Kraft was right: Belichick was a "schmuck" for ignoring the league memo to do something that had been "legal" in the past but was forbidden at the time of that particular taping and that gained the Patriots no advantage on the field and caused the franchise serious brand damage (outside New England and Patriots nation) off the field. Yes, the penalties were excessive and the cowardly product of a rookie Commissioner influenced by a media frenzy that the Patriots for all intents and purposes did nothing to stop (whether or not they could have stopped it is another subject), but that's all just la-de-da.

This x100

 

[PatsWickedPissah]

how the hell does anyone think they can quantify heart, grit, determination and courage/coolness under pressure?

this offseason is a perfect example...day after agonizing day, reams and reams of charts drawn from stat sites listing players numbers...every conceivable stat that exists. How many hundreds of posts on Michael Buchanan's "upside" citing these libraries of stats have inundated our board the past four months?...and yet, Thursday's exhibition showed that yeah, he's really cut, yeah he's really ripped, yeah, he's really athletic, yeah he really LOOKS like a stud NFL player...only one problem....he is a LOUSY excuse for an NFL level depth player, nevermind a starter on ANY defensive squad in the league.

Spot on.
While I look like a studly matinee idol, I am a LOUSY excuse for a movie actor.

 

[Danger Zone]

I happen to be an amateur statistician and I will confirm for you that 84.76% of all stats are made up.

 

[Jercules]

The NFL is obsessed with keeping technology off the sidelines (an old 60 Minutes interview with John Fox told me that much). Notice they now allow tablets that show still shots of plays... but not video, of course, even though they easily could? That's the only reason it became verboten to use a camera to do the work of a man with a pen, paper, and working knowledge of defensive signals: The fear that technology on sidelines will take the gamesmanship/guesswork out of coaching.

Despite all that, I always found the best argument to be the simple fact that the rule was not instituted before 2006 (as in, after all three Pats SB victories). Do we say none of the NJ Devils Stanley Cup wins count because Marty Brodeur was always entering the trapezoid?

 

[Mogamedogz]

I cant believe that some of you still get upset over this stuff.

Time to let it go. Almost every fan of every team will always think The Pats cheated. Period. This is like trying to convince a devote christian that god is a fairy tale, or a far right conservative that Obama isn't a Muslim. Or a far left liberal that Bush was a competent president.

It's an unwinnable argument that will ALWAYS result in lessening your enjoyment as a fan.

Just ignore that noise and enjoy the last few years of what will likely go down as the best era of Patriot football in our lifetimes. Who cares what those haters say.