Any Statisticians out there? 538 weighs in on Spygate.

The Patriots' Opponents Won't Let Spygate Die, But Did It Really Matter? | FiveThirtyEight

It went over my head a bit with talk of one tailed-T tests etc. I am not a statistics person. Just thought this was interesting (yes, I rolled my eyes too).

It's not a terrible article, though it punts at the end.

It also ignores the fact that there's a very good reason why the Patriots struggled in some of their post-Spygate losses (viz., injuries to key offensive players).

It acknowledges the issue of too many variables but yeah…. there are WAY too many variables when you talk about the postseason. Other teams should be studied as well.

Too much random stuff happens to derive any cause and effect conclusions. One guy kicks a field goal in a snow storm and another guy catches the football with his helmet. 2 things that have to happen to even have this discussion.

Getting somewhat off topic....

But I find it fascinating how people just seem to think that the signals were picked-up upon so that the offense knows the coverage before the snap. As if the videotape was broken down mid-game. That's impossible. People have come up with ridiculous scenarios to explain this discrepancy too (hidden in-helmet radio's, or breaking down the video at halftime so the info can be used 2nd half, lol).

In reality, the tape is simply used so the coaches can know -- after the game -- what coverage the opposing D was attempting to execute; that way, they can do a faster job of breaking things down. I have no problem with this act being illegal, or the Pats getting punished for it. I just think the significance of it is being totally blown out of proportion....even from the people, like the author of the above article, who are trying to question Spygate's significance.

Spygate tapes simply streamlined the scouting process

Ok, here's my one sperglord post for the year.

Spygate: The taping of defensive signals from the sideline in 2007 after a memo forbidding that was sent out during the offseason.

I cannot think of better criteria to measure offensive performance than yards per game and points per game.

The years prior to spygate:

2001 - 19th in ypg, 6th in ppg.
2002 - 21st in ypg, 10th in ppg.
2003 - 17th in ypg, 12th in ppg.
2004 - 7th in ypg, 4th in ppg.
2005 - 7th in ypg, 10th in ppg.
2006 - 11th in ypg, 7th in ppg.

The post-spygate years:
2007 - 1st in ypg, 1st in ppg.
2008 - 5th in ypg, 8th in ppg.
2009 - 3rd in ypg, 6th in ppg.
2010 - 8th in ypg, 1st in ppg.
2011 - 2nd in ypg, 3rd in ppg.
2012 - 1st in ypg, 1st in ppg.
2013 - 7th in ypg, 3rd in ppg.

The Patriots offense has been objectively better every single year with the exception of Brady's injury year after spygate. Which is why I cannot be bothered to agree or even pay attention to the '0 SB victories post spygate, lolz' meme.

What's striking is that in all of our SB victory years our YPG ranking was lower, and in two cases significantly so, than our PPG ranking. What does that tell you?

That our success in those years had a lot to do with an awesome defense and special teams that put our offense in a lot more short field situations.

Starting in around 2005 our pass defense began to really struggle, rebounded for a few years and then just tanked. Our awful defense over the last few years has been the achilles heel of this team not some overblown rules violation.

I don't condone what the team did but anytime anyone tries to say all of their success was due to taping defensive signals I lose all ability to respect their football opinion.

My big problem with the "Spygate mattered because they haven't won a SB since they got caught" argument is that it doesn't bother to analyze HOW the Patriots fell short on a year by year basis, and simply points to a championship drought of X number of years to validate it's point. Furthermore, if Spygate was as big a deal as they say it is, then explain why the Pats failed to even make the Superbowl during the 02, 05, and 06' seasons?

fnordcircle said:

Ok, here's my one sperglord post for the year.

Spygate: The taping of defensive signals from the sideline in 2007 after a memo forbidding that was sent out during the offseason.

I cannot think of better criteria to measure offensive performance than yards per game and points per game.

The years prior to spygate:

2001 - 19th in ypg, 6th in ppg.
2002 - 21st in ypg, 10th in ppg.
2003 - 17th in ypg, 12th in ppg.
2004 - 7th in ypg, 4th in ppg.
2005 - 7th in ypg, 10th in ppg.
2006 - 11th in ypg, 7th in ppg.

The post-spygate years:
2007 - 1st in ypg, 1st in ppg.
2008 - 5th in ypg, 8th in ppg.
2009 - 3rd in ypg, 6th in ppg.
2010 - 8th in ypg, 1st in ppg.
2011 - 2nd in ypg, 3rd in ppg.
2012 - 1st in ypg, 1st in ppg.
2013 - 7th in ypg, 3rd in ppg.

The Patriots offense has been objectively better every single year with the exception of Brady's injury year after spygate. Which is why I cannot be bothered to agree or even pay attention to the '0 SB victories post spygate, lolz' meme.

What's striking is that in all of our SB victory years our YPG ranking was lower, and in two cases significantly so, than our PPG ranking. What does that tell you?

That our success in those years had a lot to do with an awesome defense and special teams that put our offense in a lot more short field situations.

Starting in around 2005 our pass defense began to really struggle, rebounded for a few years and then just tanked. Our awful defense over the last few years has been the achilles heel of this team not some overblown rules violation.

I don't condone what the team did but anytime anyone tries to say all of their success was due to taping defensive signals I lose all ability to respect their football opinion.

Click to expand...

Thank you for doing the research for me - was just about to post those rankings. I'm an actuary, an occupation which deals with statistical measures a lot. All the statistical tests in the world won't help much if the underlying data or process is flawed.

They're using Vegas point scored expectations as their baseline testing process. Logic would dictate that a team that is highly ranked in points scored (2007 - 2013) is less likely to exceed expectations than a team in generally in the middle of the pack (2000 - 2006). There is no place to go but down when you're at the top. This is especially true come playoff time - once all Vegas gamblers have a full season to understand how good the team's offense is, and when teams are playing tougher defenses.

They need to run a playoff comparison of all highly ranked offensive teams versus their Vegas expections, and the compare the Pats' record against that, to get a possible meaningful analysis.

fnordcircle said:

you?

That our success in those years had a lot to do with an awesome defense and special teams that put our offense in a lot more short field situations.

Starting in around 2005 our pass defense began to really struggle, rebounded for a few years and then just tanked. Our awful defense over the last few years has been the achilles heel of this team not some overblown rules violation.

Click to expand...

Completely agree. That, and injuries to our key players are responsible for the majority of our playoff failures over the past decade.

I'm curious where the magical signals came from against the Panthers. They last played in 2001 when the Panthers were 1-15 with a different HC and QB. So, if the theory is, the Patriots played an opponent, stole their signals, and then used those in the next game, how does that apply to CAR?

Aside from some of the very valid points above, the flaw I am finding in this statistical analysis is that it presumes all else is the same in all the before and after games, and that is not the case. While in theory using Vegas odds accounts for all those other variables, that is not necessarily so.

In terms of using stats to 'prove' something regarding the effect of taping, that's a flaw that would/should never stand up in an unbiased hypothetical deductive theory. There are so many variables it's like saying because I wore my blue shirt every time they won and another shirt when they lost, that the blue shirt caused the victory. There's a mile-wide difference between causation and correlation.

Statistically speaking, an NFL team in the salary cap/free agent era that excels for a few years should be expected to soon regress. Examples would be what happened to the Rams or Bucs after their championships, or to a lesser extent the Ravens, Steelers, etc. Same goes for teams that were very good but didn't win it all, like the 2004-09 Chargers or the 2003-07 Seahawks.

End result is that most people will use stats to fit their agenda, regardless of outcome. If the Patriots had regressed the way the Rams, Bucs, Raiders and others did, then that would be their 'proof' of the advantage. If the Pats don't regress but remain highly competitive, but don't win it all, then that too is their 'proof' of the advantage. If the Pats do win another championship then it is still their 'proof', because it is fewer championships. And if the Pats win just as much or more, then those same people will cite that as 'proof' that they must be doing something nefarious, because to win that often is such a large statistical anomaly that there can be no other explanation.

In other words, regardless of the outcome, the conspiracy theorists will always have some sort of explanation that backs up their bias.

In Little League our coach taught us the art of signal misinformation -pretend to be signaling in one play -but doing something else.

I guess 31 NFL teams were:

1) Ignorant that their signals were at risk
2) Too stupid to gain an advantage over their opponents by using misinformation in their signal calling
3) Too stupid to change their signals

League is full of Spybabies whining over Crygate.

Who cares? Over a decade of dominance, that's seen many a conference rival come and go while the Patriots remain. A post-salarycap Dynasty with 5 Superbowl appearances, 8 AFC Championship appearances, and more wins that were just embarrassing to our opponents than I have ever seen another team in league history match.

If you were another team you would want something imaginary to cry over too.

I haven't studied how the whole taping process works as well as most on here have, so I was hoping one of my fellow posters might be able to clarify why Anderson's memo designated the sidelines as a specific area to prohibit taping (this is, assuming I understood the ruling correctly), did it provide any iota of a competitive advantage, and if so, of what kind? I recognize that it was legal prior to the memo, but I wanted to get my facts straight on this whole issue, even if I am positive that's it's the most overblown sports scandal in recent memory.

I tried looking this up, but all the articles I've found so far seem to be written during the fallout of the scandal, and many of them seem to be penned by writers with axes to grind.

We are the Yankees of the NFL. Its a good thing. Also, we play in a league with a hard cap.

The memo was sent to the entire league, clearly they thought that taping signals was rampant or that would not have been the case. The patriots were guilty of taping Jets signals for about 1/4 of one game and that's it, a 25.00 $ fine and taking away Belichick's first pick in his fantasy league would have been the appropriate punishment

Efstratios said:

My big problem with the "Spygate mattered because they haven't won a SB since they got caught" argument is that it doesn't bother to analyze HOW the Patriots fell short on a year by year basis, and simply points to a championship drought of X number of years to validate it's point. Furthermore, if Spygate was as big a deal as they say it is, then explain why the Pats failed to even make the Superbowl during the 02, 05, and 06' seasons?

Click to expand...

Don't forget the 5-11 season in 2000.

I'm a statistician (demographer) irl

Like most 538 articles this is complete crap. First off, the playoff record is not subject to random chance for a lot of reasons both substantive and methodological.

Broadly speaking, the issue with using a football game "win" as the unit of analysis is that a win is highly random and really the result of tens or hundreds of random or near-random events. It's why judging quarterbacks on wins or Super Bowls is a really dumb way to judge them. A win can turn on a single completely random fumble (fumbles have been shown to be completely random in who recovers them). Games aren't the level of analysis for football - individual plays are. And even those are incredibly random compared to a basketball possession or baseball push, which makes statistical analysis of football highly susceptible to vagaries.

Points are also not a good way to analyze football. Points are awarded in units of 2 (rarely), 3 (often), 6 (rarely), 7 (often), or 8 (rarely). They're not regular. It's why I object highly to the "net expected points" metric that has become somewhat en vogue - football points are an ordinal and not continuous measure, compared to baseball runs for instance.

Moreover, using Vegas spreads as a way to analyze anything at all is incredibly stupid. Vegas lines don't move based on what they think the score will be or anything. They move on where the money is going in order to get a profit. The Cowboys, Patriots, Giants, and other popular teams are more likely to have money move their way than other teams, for instance. People will bet on the Patriots to win even when that isn't a sensible bet because Patriots fans have money and will bet and are confident in those bets.

I hate Neil Paine. He likes to use freshman level econometric analysis as a way to show he's a scientist or something, using things like p-values and t-tests - incredibly simplistic in the world of statistics but arcane sounding to most non-statisticians - to make it seem like he's a real analyst.

This research has a small sample size, a cherry-picked dataset (which he admits, at least), and the methodological and substantive basis of the study is underwhelming at best. If you want, print this out and use it as toilet paper. That's all it's good for - and I'll say that not as a Patriots fan, but as a statistician.

primetime said:

I hate Neil Paine. He likes to use freshman level econometric analysis as a way to show he's a scientist or something, using things like p-values and t-tests - incredibly simplistic in the world of statistics but arcane sounding to most non-statisticians - to make it seem like he's a real analyst.

Click to expand...

Whoever this Mr. Paine is, hating him just because you disagree with him is unhealthy.

shmessy said:

Whoever this Mr. Paine is, hating him just because you disagree with him is unhealthy.

Click to expand...

************ what's with patsfans and being super internet sensitive