"Patriots easily feature the best long distance plan"

[solman]

Did I make that claim? Point out anywhere in this entire thread where I stated "The Patriots can be expected to score 78.5% of the time when they start within their own 20," or anything remotely like that. I compared the Patriots' performance - 11 of 14 - with the rest of the listed teams - 7 of 54.

Lets recap:

1. You quoted Kirwan saying that "The most impressive aspect of the Patriots' long drives all season is the fact that they have had 14 possessions start inside the 20 and they have scored a touchdown 78.5 percent of the time."

2. I wrote: "Its a very impressive statistic, but its bogus in the sense that it results primarily from the small sample size."

3. You wrote: "Btw, do you know what sample size would make the comparison valid, from a purely statistical standpoint? I do..."

4. You followed up your own post with: "Hmm, no response, I guess...gotta love it when folks post things like "sample size is too small" when they don't know what they're talking about"

Now you are surprised that you have been called an ass?


78.5% is bogus (i.e. has a misleading appearance) primarily because of its small sample size.

If he had used a larger sample (which obviously he can't) he would have wound up with a much more reasonable (and dramatically lower) number.

In response to this (rather obvious) statement which required no math, you have decided to break out your high school text book and question whether or not I know what I am talking about.

And please don't try to tell me that Kirwan's statistic makes no implication about future performance. Of course it does. That's specifically what Kirwan is trying to do as per the first paragraph of his article. "The hype surrounding the Patriots can be tough to listen to every day of the week. Are they going 16-0? Are they the greatest offense of all time? Can anyone stop them? None of the hype really addresses the question of just how the Patriots could beat the Colts or vice versa."

He's trying to use these statistics to make predictions about future Patriot's performance. We all are.

You continue with your "bogus" a "small sample size" mantra,

I made one initial statement "Its a very impressive statistic, but its bogus in the sense that it results primarily from the small sample size."

On that sole basis you decided that I don't "know what I am talking about".

If you didn't want to debate the accuracy of my statement, why did you challenge it?

I agree that it would be prudent not to use mathematics to challenge statements about bogosity, but that is exactly what you did, isn't it?

You churn out technical-sounding language like "which hypothesis I'm testing", "strength of data", etc. that have no relevance.

No relevance? Now you are just showing your ignorance.

You specifically asked me "do you know what sample size would make the comparison valid".

The answer depends on your hypothesis and the strength of the data.

If your hypothesis is "The Patriots will score TDs from inside their own 20 more frequently than the Colts", that will generate a different answer than "The Patriots can be expected to score 78.5% of the time when they start within their own 20".

The more strongly your data supports the hypothesis, the fewer samples are needed. If team A scores 40% of the time and team B scores 20% of the time, then you need more samples than if team A scores 80% of the time and team B scores 10% of the time.

But you didn't answer my decidedly relevant questions. Instead you went on to make your own assumptions and misapply the same branch of mathematics that you use to attack others.

I love it, as if this is a methodology I invented.

You chose to use chi-square (which assumes trials are independent and that each type of trial has an identical a priori probability of each result) on a data set which is known to obey neither criteria (and which, in fact, departs from those criteria in spectacular fashion).

The problem lies not in the chi-square test, but your choice to apply it here. I generously provided you with an example of how using chi-square on the same data set produces an obviously nonsensical result.

I also did you the favor of telling you that you are an ass.

You are most welcome.

 

[Rainy-Day Patriot]

"in the situations from which their sample was drawn"

Think about the inference that you are trying to make. The inference is not about the sample, but rather it is about what the sample represents--something about the Patriot's ability to score via long drives. If we were only concerned with the samples, no significance hypothesis testing would be necessary; instead, we would look at the observed differences and call it a day.

Here's a fact about the world: improbable things (in general) will probably happen. That's no big deal. I throw my sharpie at the wall and it makes a mark on one particular spot on the wall. Gee, what was the probability that I would hit that particular spot? Close to zero.

The trick is to call your shot. You have a theory about the Patriot's ability to score via long drives. Now, go out and test it! Don't observe a perhaps freak occurrence and retrospectively start running your little chi-squares. Your sample is not random. You found a striking sample and ran with it.

 

[FlyingElvis75]

Lets recap:

1. You quoted Kirwan saying that "The most impressive aspect of the Patriots' long drives all season is the fact that they have had 14 possessions start inside the 20 and they have scored a touchdown 78.5 percent of the time."

2. I wrote: "Its a very impressive statistic, but its bogus in the sense that it results primarily from the small sample size."

3. You wrote: "Btw, do you know what sample size would make the comparison valid, from a purely statistical standpoint? I do..."

4. You followed up your own post with: "Hmm, no response, I guess...gotta love it when folks post things like "sample size is too small" when they don't know what they're talking about"

Yes, after it was clear you came onto the board, ignored my response (posted elsewhere), and couldn't "back up" your claim. And somehow, being called out for posting BS entitles you to ad hominem attacks...right.

78.5% is bogus (i.e. has a misleading appearance) primarily because of its small sample size.

If he had used a larger sample (which obviously he can't) he would have wound up with a much more reasonable (and dramatically lower) number.

And what number, approximately, would that be? Seriously, aside from the fact that NOBODY insisted that their future conversion rate would be 78.5%, what evidence - anywhere - do you have that their performance from inside the 20 would be "dramatically lower"?

In response to this (rather obvious) statement which required no math, you have decided to break out your high school text book and question whether or not I know what I am talking about.

Which you clearly don't. On the contrary, the notion that "the sample size is too small" does require math, which you a) apparently didn't bother to do, and b) misunderstood when it was shown to you.

If I score a TD and a FG on two drives, I have 10 points in two drives, average 5. Should I expect to score an average of 5 points on all of my drives, then? Of course not; with a sample size of 2, given outcomes of 7 and 3 points, there is literally no significant difference between that outcome and zero points per drive, as a simple t-test would reveal. But of course, you knew that, right, since it's the first test that anyone with the slightest background in statistical methods learns.

You continue with your ruse of knowing anything about what you claim to be talking about.

And please don't try to tell me that Kirwan's statistic makes no implication about future performance. Of course it does. That's specifically what Kirwan is trying to do as per the first paragraph of his article. "The hype surrounding the Patriots can be tough to listen to every day of the week. Are they going 16-0? Are they the greatest offense of all time? Can anyone stop them? None of the hype really addresses the question of just how the Patriots could beat the Colts or vice versa."

He's trying to use these statistics to make predictions about future Patriot's performance. We all are.

Of course we are. But some of us are making valid predictions, and some are not, and some are trying to undermine their misinterpretation of valid predictions when there are absolutely no factual grounds for doing so.

I made one initial statement "Its a very impressive statistic, but its bogus in the sense that it results primarily from the small sample size."

On that sole basis you decided that I don't "know what I am talking about".

If you didn't want to debate the accuracy of my statement, why did you challenge it?

I challenged it. I showed you EXACTLY why it was wrong when comparing the results of Pats' drives to other teams' drives.

I'll go further, and look at the stat you explicitly have problems with: the 78.5% success rate.

No relevance? Now you are just showing your ignorance.

You specifically asked me "do you know what sample size would make the comparison valid".

The answer depends on your hypothesis and the strength of the data.

Which you failed to answer, and I showed quite clearly that the question was moot in this case. And you act as if we have a choice in identifying "hypotheses" in the context of a statistical significance test...incredible.

If your hypothesis is "The Patriots will score TDs from inside their own 20 more frequently than the Colts", that will generate a different answer than "The Patriots can be expected to score 78.5% of the time when they start within their own 20".

Nobody made the second hypothesis, and this further reinforces the impression that you are trying to BS rather than make any rational argument.

When testing any single statistic for significance, there are ONLY two hypotheses: the null hypothesis (sample result is due to random chance) and the alternative hypothesis (sample result is not due to random chance). (Just for you, I opened up my old stats textbook for the first time in over 10 years, and went to the chapter on hypothesis testing. The information above appears on page 2.)

A small sample size means you are very likely to find the null hypothesis cannot be rejected. If you have a sample size of 2 where the average is 5, there is a much greater chance that the "5" is due to random chance than if the sample size is 100 (with the same std dev). With a sample size of 14, the statistic IS significant (p < 0.001) in this case. There is no valid, objective argument for the case that this sample size is too small. In short, your complaint about "small sample size" is factually incorrect, whether you're talking about the comparison to other teams or just looking at the stat itself.

The fact that you would interpret this to mean "they are going to score 78.5% of the time" or, even worse, "they should back themselves up to within their 20 if they want to score a TD" shows even more blatantly how poor your grasp of these basic concepts is.

Nobody would expect the random person on the street to understand, but if you're going to critique numbers that you explicitly asked for ("back it up") and fling personal insults along the way, you'd better damn well know what you're talking about.

The more strongly your data supports the hypothesis, the fewer samples are needed. If team A scores 40% of the time and team B scores 20% of the time, then you need more samples than if team A scores 80% of the time and team B scores 10% of the time.

"Strength" has nothing to do with it, and researchers don't use that term because it has a specific meaning in other tests (correlation). The point is irrelevant, because even with our sample size of 14, we detect a significant difference. Small sample size affects the power of the test (probability of getting a false negative), but that's moot since a difference has been found.

But you didn't answer my decidedly relevant questions. Instead you went on to make your own assumptions and misapply the same branch of mathematics that you use to attack others.

In fact, I did, pretty clearly, answer your questions. That you fail to understand the responses, and continue to make irrelevant, mistaken, and ad hominem statements, is your failing, not mine.

You chose to use chi-square (which assumes trials are independent and that each type of trial has an identical a priori probability of each result) on a data set which is known to obey neither criteria (and which, in fact, departs from those criteria in spectacular fashion).

Do you actually understand anything you apparently copy-and-paste? Would you like to explain how, in fact, the data set "is known to obey neither criteria"? (Hint: It isn't.)

You had plenty of opportunities to make a factual case. You apparently don't understand what a significance test actually means.

You didn't mention the chi-square test until I brought it up (which, incidentally, was the first test one of my interns proposed when I presented these data to her).

You could have proposed an even more basic t-test, but you didn't (and which would also have proved you wrong, in spite of its questionable utility). If you had, you could have proposed that their future performance - in the same situations from which the sample was drawn - has a large chance of being anywhere from 100% to 54%, and a small chance of being even lower than that - but you didn't. Instead, you state that their performance would be "dramatically lower," without the least bit of evidence.

You still have yet to propose anything remotely objective to support your contention that the sample size renders any of these numbers meaningless.

Instead, you let others do the work and then fling mud when it doesn't agree with your gut reactions. Do you really want to review your other post and point out every one of the gross misunderstandings you exhibit, and your complete misinterpretation and misrepresentation of the facts?

 

[FlyingElvis75]

"in the situations from which their sample was drawn"

Think about the inference that you are trying to make. The inference is not about the sample, but rather it is about what the sample represents--something about the Patriot's ability to score via long drives. If we were only concerned with the samples, no significance hypothesis testing would be necessary; instead, we would look at the observed differences and call it a day.

You're absolutely correct - a sample is a representation of a "population," and the population in this case means (in the strictest, most theoretical sense) an infinite set of drives by the Patriots under the same conditions (opponent, personnel, game situation, starting position, etc.). If we had hard data from all those drives, we would know exactly what their performance would be. But there's essentially no way we could ever get that data.

Instead, we take a sample, and estimate the "population" characteristics based on it. If the sample is random (given the population we're talking about, I'm not sure how our sample could be any more random - but feel free to propose something else), then certain mathematical rules hold true - that's the purpose of significance testing. It gives us an estimate of what the "true" population value is, with certain caveats, and tells us how likely it is that we would get these sample results if the "true" value were something totally different. That's what a chi-square, Fisher's exact, many permutations of t-test, etc. are designed to do, each in a different way, with different types of data.

But none of that has to do with the sample size in this case.

 

[Oswlek]

Whether you want to argue about the validity of the "78.5%' statistic, how in the world is 14 too small of a sample size to base analysis on?

When you score 11 TDs on 14 possession that have to go more than 80 yards that is clearly ridiculously impressive. Are they going to keep it going at that rate? Who knows. If it was 2 in 14, the exact same sample size issues would be present. These issues are true for most stats at this point. If you want to target this one, you have to target mostof them because the season is only halfway (at most) done.

There is obviously something going on that allows the team to be that successful on long drive opportunities, and 14 is more than enough to make that judgement.

 

[solman]

Yes, after it was clear you came onto the board, ignored my response (posted elsewhere), and couldn't "back up" your claim. And somehow, being called out for posting BS entitles you to ad hominem attacks...right.

1. I responded to you the first time I saw your question, some 21 hours after you originally posted it. If you wanted to make sure that I saw your response earlier you could have PM'd me. I'm sure I fail to respond to threads all the time. Had somebody else not posted in this thread later on, I likely never would have seen it.

2. In your second post, you never asked me to back up what I wrote in that first post. Instead you asked me a math question (which we have already established is unanswerable without some critical clarifications and assumptions which you failed to provide.).

"[Edit] Btw, do you know what sample size would make the comparison valid, from a purely statistical standpoint? I do..."

3. Accusing me of not knowing what I am talking about when I haven't answered your unanswerable problem in your preferred time frame does make you an ass. If you wanted to avoid personal attacks, this was probably not the way to go.

That said, I did not mean calling you an ass to be an ad hominem attack. I called you an ass because you behaved as one, but it is perfectly possible for somebody to be an ass and correct at the same time. I provided ample information as to why your arguments were wrong, and it has nothing to do with you being an ass.

Here are a couple of things you can do in the future to avoid being called an ass:

A. Don't respond to a thread about football statistics with a math question.

B. Don't accuse people of being ignorant if they don't immediately respond to your math question.

C. If you ask a math question, and call people ignorant for not immediately responding, make sure that you have provided all of the information and assumptions necessary to provide a single, indisputable answer.

And what number, approximately, would that be? Seriously, aside from the fact that NOBODY insisted that their future conversion rate would be 78.5%, what evidence - anywhere - do you have that their performance from inside the 20 would be "dramatically lower"?

In my initial post, I noted: "For example, the Patriots have a much lower success rate when starting ON their 20. But if you ask Belichick whether he has a better chance of scoring from his own 10 or 20, you can be sure he'll take the ball on the 20."

I subsequently pointed out that the Patriots' success rate on all drives NOT inside their own 20 is 41.5%.

These numbers in conjunction with certain football facts:

1. Better field position increases the odds that a drive will result in a TD
2. A TD conversion ratio over 40% is virtually unheard of. I believe that the 2004 Colts conversion rate of 36.7% (form everywhere on the field) is the highest since 1998.

Are sufficient to demonstrate that their performance from inside the 20 would be dramatically lower.

Which you clearly don't. On the contrary, the notion that "the sample size is too small" does require math, which you a) apparently didn't bother to do, and b) misunderstood when it was shown to you.

This is like saying that hanging picture DOES require a ruler and a level. It is perfectly possible for a person with a firm grip on the underlying subject matter to determine that a number has been artificially inflated by a small number of trials WITHOUT doing any calculations. That is what I did. The absurd number of 78.5% results primarily from the fact that only 14 trials occurred (and that the results of these trials were highly correlated, not independent at all).

But seriously, is there anybody here who would actually consider betting that the Patriots will convert even 50% of such drives the rest of the season? It seems to me that we all know its a bogus statistic. There is no need to use mathematics to prove its bogosity. The is no math which can rescue this statistic from such a characterization.

If I score a TD and a FG on two drives, I have 10 points in two drives, average 5. Should I expect to score an average of 5 points on all of my drives, then? Of course not; with a sample size of 2, given outcomes of 7 and 3 points, there is literally no significant difference between that outcome and zero points per drive, as a simple t-test would reveal. But of course, you knew that, right, since it's the first test that anyone with the slightest background in statistical methods learns.

Your statistical knowledge is deficient. A simple t-test is an inappropriate test because:

1. It requires the data to conform to some probability distribution the general form of which is known a priori. "simple t-test" and "student t-test" usually mean that we assume a normal distribution. This is obviously incorrect. Although 4 point drives would be cool, there hasn't been one since FGs were set to three points.
2. The simple t-test assumes that each measurement of the same type has the same a priori distribution. For this set of data, this assumption is obviously not true. Different starting locations with different scores against different opponents with different amounts of time on the clock have different probability distributions. Since these differences are in no way uniform, you can't just ignore them. [For a concrete example of how this reduces certainty, you can repeat your previous chi square test using separate lines for each of the three comparison teams.]
3. The simple t-test requires the data to be uncorrelated, which we know to be false.

In all three cases the t-test can be reformulated to compensate for these deficiencies, but doing so requires a priori knowledge that we lack and can not reasonably obtain without introducing other problems.

You continue with your ruse of knowing anything about what you claim to be talking about.

I think I'm done here. You are repeatedly using methods out of a high school textbook without first making sure that the assumptions behind those methods are satisfied by the data. You then ignore the wealth of information that our actual knowledge of the game of football provides us with.

If you think my knowledge is deficient, so be it. I think I've clearly demonstrated otherwise.

I have only ever had one point in this discussion:

Pat Kirwan's stat (that the Patriots score on 78.5% of drives starting within their 20) is bogus.

 

[njpatsfan]

Your statistical knowledge is deficient. A simple t-test is an inappropriate test because:

1. It requires the data to conform to some probability distribution the general form of which is known a priori. "simple t-test" and "student t-test" usually mean that we assume a normal distribution. This is obviously incorrect. Although 4 point drives would be cool, there hasn't been one since FGs were set to three points.

Wow. Here we see illustrated the difference between someone reciting knowledge versus someone with true understanding of the principles.

Solman by TKO.

R

 

[FlyingElvis75]

For Pete's sake. I offered you the chance to just let it go. But you insist on somehow trying to bring your complete ignorance of the subject matter to light.

The entire point of a t-test is that the statistic is NOT normally distributed. Do you even know where the t-value in "t-test" comes from? I'm sure you could google how it's calculated, but do you actually understand it? There would be NO NEED for it if we assumed all variable values were normally distributed.

Now you're saying "the data need to be uncorrelated." Here's a news flash: a correlation requires more than one variable. There is no such thing as a correlation within ONE variable. The fact that you would even bring this up - again, only after *I* mentioned correlation - exposes your lack of understanding.

As if that weren't enough:

1. It requires the data to conform to some probability distribution the general form of which is known a priori. "simple t-test" and "student t-test" usually mean that we assume a normal distribution. This is obviously incorrect.

If this is incorrect, then every t-test ever performed in any context is incorrect. The test does NOT assume the individual observations in the sample are normally distributed - it assumes that over repeated samplings, the statistic we are measuring will assume a t-distribution (which approximates a normal distribution at very large sample sizes) if the samples are drawn from the same population. It's the Central Limit Theorem, the most basic freakin' mathematical proof that exists in statistics. The reason for Student's t's existence is to address sample size.

Seriously - do you think that when we calculate the inflation rate, personal income changes, the incidence of cancer, etc. that we assume that prices, income, cancer cases etc. conform to a normal distribution? This is a misunderstanding that student interns get over in their first week!

2. The simple t-test assumes that each measurement of the same type has the same a priori distribution.

Exactly - in essence, that the population does not change between samplings. If you had bothered to read my response to Rainy-Day, you'd actually understand what "population" means, and if you had the ability to comprehend "in the same situations from which the sample was drawn," you wouldn't be posting utter nonsense as if it had any relevance to what we're talking about.

3. The simple t-test requires the data to be uncorrelated, which we know to be false.

Again - YOU NEED AT LEAST TWO VARIABLES to produce a correlation. Running a correlation between a single variable and itself *always* results in perfect correlation, which is utterly meaningless because it is ONE VARIABLE.

A case can certainly be made that a t-test is not the most appropriate test for the data. But the reasons have nothing to do, even remotely, with the ones you posted above.

Honestly, I had started to feel bad after my last post. I was beginning to think you did have some background at one point, but were rusty and had maybe made some honest mistakes, exacerbated by your reaction to my poor tone in my third post (for which I now apologize). But after this last post, it's becoming clear that you are doing nothing but looking up terms I put up there, trying to grasp their meaning (and failing), and coming back to post even more gibberish in the hopes that it will help you save face. In fact, it's doing the opposite.

More?

According to chi-squared (as utilized by you), this proves (likelihood of error a smidge more than 1%, well within the 5% that you expressed comfort with)

a) "Error" has nothing to do with that probability, and "error" has a completely different (and important) meaning, even in this very test. I even explained to you what the probability figure represents, and you STILL misunderstood it.

b) What "I express comfort with" is not important. What the vast majority of researchers use as convention is. If you'd like to propose a different target alpha, feel free - the results will be similar.

Which of the following would explain this result:

1. The Patriots are more likely to score a TD on drives starting within their own 20 than when they have better field position
2. The use of chi square in this instance (with your methodology) produces an erroneous result.

I could understand Joe FootballFan presenting a false dichotomy. For someone who claims to know how to apply statistics in any remotely credible way to commit a fallacy like that - it completely strains credulity.

The figures we are looking at explain what happened, and what can be expected to happen under similar conditions. They don't explain why. There are literally dozens of possible explanations out there, probably the least credible of which is the idea that "math is wrong."

Do opponents with crappy goal-line defenses tend to try to pin kicks deep, rather than going for touchbacks? Are the Patriots better at wearing down defenses over long drives than other teams? Do the Patriots fair-catch more on deep punts when they feel the opposition's goal-line defense is crappy? I don't know. The numbers don't tell us why they are the way they are - no single stat does - but there are MANY possible explanations, not all of which may be in the Patriots' control.

I think we can safely choose option number 2, but if you believe otherwise feel free to send BB and email and tell him to stop trying to return kickoff returns and punts beyond the 19.

Another fallacy. If you understood what a significance test meant, you'd understand why. Or would you like to assert that any of the tests I've proposed is directional? If so, write up a proof - you'll be hailed as the greatest statistical genius of the past 50 years.

On the other hand, publish a research article containing either of the fallacies above, and you're the laughingstock of the research community.

It is perfectly possible for a person with a firm grip on the underlying subject matter to determine that a number has been artificially inflated by a small number of trials WITHOUT doing any calculations. That is what I did. The absurd number of 78.5% results primarily from the fact that only 14 trials occurred (and that the results of these trials were highly correlated, not independent at all).

You have presented no evidence that you have anything remotely approaching a "firm grip" on the subject matter. You made a statement asserting that the sample size was too small, which I *thoroughly* addressed above. I even addressed the (large) probability that the conversion rate would be lower after repeated trials, and I gave you the 95% confidence interval - which you could easily have done, even on the first page, if you had any idea what you were talking about.

You could even have proposed a binomial test, which would reveal that the 11-for-14 is not significantly different from 50-50. You could have pointed out that, given the t-test results, the "true" value even has a chance of being below 54%. But you didn't, which speaks volumes, considering it would be a fact that might support your case.

Instead of addressing the mathematical facts I posted, you come back with inaccurate and completely ignorant claims about the validity of assumptions that you clearly don't understand. And you continue to maintain not only that a) you have a good grasp of statistical methods - which, by the way, would not have been brought up in this thread had you not insisted that I "back it up" - but b) that your original statement, now disproven in multiple ways, is true?

On top of that, you have the gall to call my "statistical knowledge," for which I am professionally employed, "deficient"? Unbelievable.

Finally:

That said, I did not mean calling you an ass to be an ad hominem attack. I called you an ass because you behaved as one, but it is perfectly possible for somebody to be an ass and correct at the same time. I provided ample information as to why your arguments were wrong, and it has nothing to do with you being an ass.

Here are a couple of things you can do in the future to avoid being called an ass:

A. Don't respond to a thread about football statistics with a math question.

B. Don't accuse people of being ignorant if they don't immediately respond to your math question.

C. If you ask a math question, and call people ignorant for not immediately responding, make sure that you have provided all of the information and assumptions necessary to provide a single, indisputable answer.

Point taken. I can see how one could view my early responses as "baiting," and for that I apologize. I had hoped to get into an intelligent debate on the merits of such an analysis, but obviously it didn't turn out that way. I take as much responsibility for this devolving into flaming as anyone else.

And here's my advice to you: Just say "I don't know" if you don't know.