# The Law of Large Numbers vs. KeithS, Eigenstate and my other TSZ critics

June 24, 2013 | Posted by scordova under Humor, Mathematics |

I went through a great deal of trouble to contest the idiosyncratic claim of a critic at TheSkepticalZone who said:

if you have 500 flips of a fair coin that all come up heads, given your qualification (“fair coin”), that is outcome is perfectly consistent with fair coins,

This critic (who goes by the handle “Eigenstate”) would probably keep singing the same tune if we were dealing with 500,000,000 fair coins. I said this was wrong, and KeithS disagreed and demanded I make a retraction. See his comments here in SSDD: a 22 sigma event is consistent with the physics of fair coins?.

I insisted the expectation value of 50% heads has to be respected, that if a theory predicts a certain expectation value, multi-sigma deviations from that expectation are reasonable grounds (not absolute grounds) to reject that theory as an explanation of that phenomenon. Yet KeithS keeps swearing by the fact (which I don’t dispute, nor never have disputed) that “all coins heads” is no more improbable than any other sequence. I never said otherwise the original post, Siding with Mathgrrl on a point, that generated the current firestorm of threads and comments.

But the fact that “all coins heads” is no more improbable than any other sequences does not negate the fact that all-coins heads is statistically inconsistent with the hypothesis of a fair coin and a random process acting on the fair coins. At issue is the fact all coins heads has a statistical property, namely, it is maximally deviant from the expectation value.

I never said “all heads coins” was more improbable than any other sequence, but I did say the following, that seemed to go in one ear and out the other over at TSZ:

For example, consider if we saw 500 fair coins all heads, do we actually have to consider human subjectivity when looking at the pattern and concluding it is designed? No. Why? We can make an alternative mathematical argument that says if coins are all heads they are sufficiently inconsistent with the

Binomial Distributionfor randomly tossed coins, hence we can reject the chance hypothesis.

They failed to acknowledge I used the Binomial Distribution. I then spelled it out for them what that meant in terms of standard deviations and expectation values in SSDD: a 22 sigma event is consistent with the physics of fair coins?.

But my critics were not merely content to hear me criticize some of Bill Dembski’s work, they wanted find fault where there was none, because the fact I might have a legitimate point is intolerable since creationists supposedly don’t like science and math.

Anti-ID critics have propensity to :

1. misread

2. misattribute

3. mischaracterize

4. misstate

5. render the most uncharitable interpretation of what is said

6. and when called on their uncharitable readings and errors, they compound their errors because of a determination to save face

Over at TSZ, I’ve happily offered statements, and then retracted errors in my calculations (see my post on the 2nd law and you’ll see I welcomed correction of my misunderstanding of the Liouville theorem). I’m grateful for the interaction with critics because:

1. they do correct errors

2. they do educate

3. they help us learn to state our points in ways less likely to be misread in the future

So, I do think the exchange is valuable. But well, in contrast, for people like KeithS, in an attempt to save face, they just say even more idiosyncratic things and keep demanding I make a retraction. I told KeithS: “No DICE”.

From Wiki on Law of Large Numbers:

In probability theory, the law of large numbers (LLN) is a theorem that describes the result of performing the same experiment a large number of times. According to the law, the average of the results obtained from a large number of trials should be close to the expected value, and will tend to become closer as more trials are performed.

The LLN is important because it “guarantees” stable long-term results for the averages of random events. For example, while a casino may lose money in a single spin of the roulette wheel, its earnings will tend towards a predictable percentage over a large number of spins. Any winning streak by a player will eventually be overcome by the parameters of the game. It is important to remember that the LLN only applies (as the name indicates) when a large number of observations are considered.

Say what? All the calculations I made regarding expectation values might actually be meaningful!

Also from Wiki:

For example, a single roll of a six-sided die produces one of the numbers 1, 2, 3, 4, 5, or 6, each with equal probability. Therefore, the expected value of a single die roll is

According to the law of large numbers, if a large number of six-sided dice are rolled, the average of their values (sometimes called the sample mean) is likely to be close to 3.5, with the accuracy increasing as more dice are rolled.

It follows from the law of large numbers that the empirical probability of success in a series of Bernoulli trials will converge to the theoretical probability. For a Bernoulli random variable, the expected value is the theoretical probability of success, and the average of n such variables (assuming they are independent and identically distributed (i.i.d.)) is precisely the relative frequency.

For example, a fair coin toss is a Bernoulli trial.

When a fair coin is flipped once, the theoretical probability that the outcome will be heads is equal to 1/2. Therefore, according to the law of large numbers, the proportion of heads in a “large” number of coin flips “should be” roughly 1/2. In particular, the proportion of heads after n flips will almost surely converge to 1/2 as n approaches infinity.Though the proportion of heads (and tails) approaches 1/2, almost surely the absolute (nominal) difference in the number of heads and tails will become large as the number of flips becomes large. That is, the probability that the absolute difference is a small number, approaches zero as the number of flips becomes large. Also, almost surely the ratio of the absolute difference to the number of flips will approach zero. Intuitively, expected absolute difference grows, but at a slower rate than the number of flips, as the number of flips grows.

No where in that wiki article is anything there that can be remotely construed to defend statements like:

if you have 500 flips of a fair coin that all come up heads, given your qualification (“fair coin”), that is outcome is perfectly consistent with fair coins,

Eigenstate criticizing Sal

Look at this graph of dice rolls and how large numbers of trials converge on expectation value of 3.5 (a similar feature will emerge with fair coins converging on expectation value of 0.5 heads). These are principles that KeithS and eigenstate don’t like discussing in their determined attempt to misread and disagree with everything I say, even textbook statistics. They will compound their errors because anti-ID critics have a propensity to never admit error, and will argue to save face at all costs, and this will lead to some very entertaining reading. 🙂

But alas, KeithS and others refuse to acknowledge these considerations, and worse he demands I make a retraction as if I’m some sort of mathematical heritic. No Dice, KeithS.

Same Silliness, Different Darwinist (SSDD).

ADDENDUM

At the request of Elizabeth Liddle, I’m highlighting her comment:

Look at the use of the word “consistent”. Eigenstate used it to mean “compatible with the stipulation that the coin is fair”. Sal interpreted it to mean “compatible with the conclusion that the coin is fair”.

Interpreted the first way, Eigenstate is correct. Interpreted the second way, Sal is correct.

I confess I don’t quite understand what Elizabeth means, but out of my great respect for her, I’m highlighting her comment.

### 36 Responses to *The Law of Large Numbers vs. KeithS, Eigenstate and my other TSZ critics*

### Leave a Reply

You must be logged in to post a comment.

Anti-IDists will generally say whatever it takes to disagree with IDists, no matter how foolish it makes them sound. They will also do everything they can to characterize IDists in a negative light.

It’s not just a debate for many of them; it’s an ideological and cultural war.

I don’t think so Sal 🙂

In my view, the question simply hinges on how we respond to your assurance that the coin was fair, and fairly tossed.

If we can trust you, absolutely, then we have actually ruled out, a priori, Dishonesty/Design, and must accept that the coin was fairly tossed.

However, this is never, with respect, the case! So we also have to factor in the prior probability that you are telling the truth when you say the coin was fair.

That is what allows us to confidently, if reluctantly, conclude, however unlikely we thought it was you were dishonest before you tossed the coins, you cheated.

That’s really all this storm in a teacup is about 🙂

eigenstate took you at your word.

The rest of us were prepared to doubt you 🙂

William:

This may or may not be true, but it is not a legitimate deduction from this particular squabble.

Inferential statistics is a complex and subtle art, and it’s perfectly possible for two people to be absolutely correct, while nonetheless vehemently disagreeing. This is simply because “probability” means different things in different context, and people simply don’t notice which sense they, or the other, are using it in.

So no need to stoke up the culture war on this occasion. Everything’s fine.

“So no need to stoke up the culture war on this occasion. Everything’s fine.”

Elizabeth, are you not involved in any way in ‘the culture war’ (Anglo-American) on your site ‘The Skeptical Zone’? If not you, then how many others on your site are actively involved?

This gets to the label of ‘Darwinist,’ which you recently called yourself, before accepting that it might not be appropriate for you when considered as ideological.

W. Murray’s #1 here is of course ridiculous. Just switch ‘anti-IDists’ for ‘IDists’ and the charge is equally valid and easily demonstrated.

Are you trying to suggest, Elizabeth, that there actually is *no culture war* at all? Or that you are just protecting innocent scientism from unjust criticism (‘scientist’ and agnostic that you are), while your IDist adversaries commit scientism at the same or higher scale?

Without its opposition to UD, TSZ would like not have survived until now; your site is that closely tied together!

I’m generally not interested in the probabilism IDists display and it doesn’t disturb me or seem that important as you and others at TSZ seem to make it. You seem to realise that taking the probabilistic line, IDists are playing right into your hands.

What I’m suggesting, Gregory, is that this little dust-up has nothing to do with a cultural disagreement (although the heat generated almost certainly has) but with lack of clarity between (and within) contributors as to what is meant by a “probability”.

Once that is clarified, as I have attempted to do on the threads here, also at TSZ, the disagreement should vanish. Everyone agrees that if

actuallysaw Sal toss 500 Heads in a row, that he’d be cheating in some way.Everybody also agrees that the probability of getting any sequence is no higher than getting 500 Heads, and therefore a sequence just as improbable as 500 Heads happen every time you toss 500 coins.

Everybody also agrees that this isn’t the point – the point is that 500 Heads is part of a very small set of Special sequences and that this enables us us conclude with virtual certainty that Sal had cheated.

Everybody, I think agrees that the Laws of Physics do not forbid anyone from tossing 500 Heads in a row.

I think everybody agrees that, although unlikely, it’s as likely to happen next time I toss, than at any time in the next 2^500 times I do it, by which time I shall be an extremely elderly lady.

Everybody is right. But everybody is also slightly wrong, because nobody (well, not before the balloon went up) was paying due attention to the key issue which is how we respond to Sal’s assertion that the coin was fair.

If that is a God’s Eye view, an absolute truth, then clearly we must conclude that we were witness to an extremely rare event. But as Sal doesn’t, I don’t think, have a God’s Eye view, we must conclude that he was lying. (I speak in hypothetically of course; my faith in Sal remains unaltered by his fictional account of tossing 500 Heads :))

The coins are fair, that was the hypothesis.

In my original post, there was no mention that the observed coins were tossed!We compare the pattern against the fair tossing hypothesis, and then reject the fair tossing hypothesis as the mechanism of pattern generation. We don’t reject the fair coin hypothesis, because that was assumed and can also be tested empirically using (no surprise), the law of large numbers.These nuances in the givens I provided are important, and we wouldn’t have so many misunderstandings if they were actually noticed and accounted for.

I’m sorry Elizabeth, but much of what you wrote criticizes an argument and scenario I didn’t state.

What you said is still important and valuable, nonetheless. Thank you for you comments.

Sal:

Ah! Well there’s another possibility that we should have taken into account – the probability that the coins were not tossed!

Well, dangerous to assume a fair coin, Sal. After all, if you had tossed them, and got some sequence extremely unlikely under the assumption of a fair coin, we’d have to conclude that the assumption was incorrect, wouldn’t we?

I would.

No need to apologise, Sal, you did nothing wrong. But it serves as a really good illustration as to how we need to factor in our priors (what Sal meant; whether I thought what Sal meant was what he meant, etc) when making an inference that there was Something Funny about that sequence of Heads.

Yes, I think it is 🙂 I think we should all have a nice cup of tea now.

And for yours 🙂

Not to ruin the 🙂 party, but I am still have a bit of distress over this thread because what it has taught me. If I read the nature of God correctly, He will never add a design signature to nature that is as

clear and easily understoodas the impossibility of randomly flipping a fair coin 500 times and getting all heads.If that is the case, the people who believe that 500 heads in a row is possible are forever lost, and will never come to God. That is indeed sad.

JDH,

God doesn’t compel anyone to believe – we have free will. It is always our choice to believe. Free will can deny anything, even the clear and obvious implication of what 500 heads in a row means.

Let’s say that we took heads to mean 1, and tails to mean 0, and we flip a number of coins. The result, in ASCI, happens to be “this is god proving to you that I exist by inserting this message into your coin tossing experiment”, that particular string of coin tosses would be no more improbable to these materialists than any other, and it would be just as casually dismissed as 500 straight heads.

Lizzie wrote trying to reconcile all things,

Sorry for raising a quibble. I think the Laws of Physics do forbid someone from tossing 500 Heads in a row. Because I always consider the Laws of Physics to be about what is possible in the real world including statistics. ( i.e. Second Law of Thermodynamics is purely a statistical law. )

I fully agree that the laws of mathematical and consideration of theoretical outcomes ( not based in the real world ) admit the possibility that a theoretical fair coin could possibly be tossed 500 times and come up heads every time. But this theoretical possibility has nothing to do with the real world.

JDH: perhaps you would like to read my rather overlong comment here and cross posted at TSZ.

I am trying to make the point that trying to argue that the Laws of Physics forbid 500 Heads, is somewhat meaningless. No, I’ll go further – wrong 🙂 There is nothing in Physics that will stop that last coin going down Heads, to ensure that there is at least one tail. And we know that, because, if it were the case, the probability of each coin going heads or tails would start to alter as a function of the previous throws, and we know this is not the case. Let’s take a row of 20 heads, which is fairly, but not prohibitively unlikely. If it were the case that 20 heads was frowned on by the Laws of Physics, we’d expect that the once you’d thrown 10 heads, some additional force would kick in, making it more likely that subsequent coins would land tails, and “preventing” 20 heads. We can demonstrate that this is not the case – coin-tosses are independent of previous coin-tosses. Indeed Shannon Information Theory is predicated on this assumption!

You are correct in your inference, but I think you’ve hit the wrong nail on the head (as has Granville). The reason we can infer Design (or skulduggery, or that the coins weren’t actually tossed but laid in a row by hand, or whatever) isn’t because the Laws of Physics prevent anyone tossing 500 heads. It’s because we know, a priori, that coin-tossing skulduggery/Design is much more common (and therefore much more likely) than throwing 500 Heads with a fair toss and a fair coin.

So given the two options, we choose Design as the more likely option. This is perfectly valid.

John 20: 24-29

🙂

My comment from last night still applies:

KeithS,

Would it make you happy if I said:

If Eigenstate never said something to that effect, but I was first to make that assertion at UD, you guys would never let me hear the end of it.

I won’t agree with that statement. It’s idiosyncratic at best, and erroneous and silly at worst.

The law of large numbers is in opposition to that statement. No Dice, KeithS.

Elizabeth

Your comment at #5 is on the surface real nice, but it has been drenched in relativism. What happened to the law of the excluded middle? Things can not be true and false at the same time, it is either or…. not both, care to revise and make a standpoint on either or? I’d really appreciate it.

Sal:

Do you mean with the statement:

?

Again, this may be semantics. I’d say that 500 heads of a flipped fair coin is indeed “consistent” with fair coins, but that it is vanishingly unlikely under the null that fair coins were indeed fairly flipped.*

Again, you are both correct within the intended meaning of your respective words. And I’m sure that both keiths and eigenstate agree with you that 500 Heads is vanishingly unlikely under the null of fair flips of fair coins.

*Although I’d cite the binomial theorem rather than the Law of Large Numbers – after all, nothing in the Law of Large Numbers would make alternating HTHTHTHTHTHT improbable, yet I’m sure you would be just as convinced that the flipping wasn’t random, wouldn’t you?

Two statements can be correct, but use the words in different ways. Consider:

Time flies like an arrow

Fruit flies like a banana.

Both are true, but if the speaker of one, meant “flies” as in the other, each might vehemently accuse the other of making a false statement!

That is what I think is happening here – over and over. Look at the use of the word “consistent”. Eigenstate used it to mean “compatible with with the stipulation that the coin is fair”. Sal interpreted it to mean “compatible with the conclusion that the coin is fair”.

Interpreted the first way, Eigenstate is correct. Interpreted the second way, Sal is correct.

And don’t get me started on the word “random”…. 😀

If we sampled every other coin, the expectation value should still be 0.5 heads for every other coin. In the case of HTHTHT…. it is not, therefore it violates the law of large numbers as well. You just have to look at the issue a little deeper to see the violation.

The notion of taking samples at regular intervals is typical in science, so sampling coins at regular intervals will detect violation of the law of large numbers.

Nevertheless, I didn’t say all coins heads is a necessary condition to reject the chance hypothesis, it is merely a sufficient condition, and that is agrees with my objection to:

Sal,

Eigenstate wrote something that was correct.

You wrote an entire OP mocking him (“Same Stuff, Different Darwinist”), claiming that he was wrong, and quotemining him in the process.

It turns out that he was correct. Even you even seem to acknowledge that. His statement is true, exactly as he wrote it.

That means your entire OP is based on a falsehood. The decent thing to do is to acknowledge your error, affirm that eigenstate was correct, and retract your claim.

Why is that so hard?

P.S. Throughout this entire discussion, I’ve been saying consistently that an “all heads” outcome requires an explanation. Don’t pretend otherwise. If you doubt me, see this comment from three days ago.

Sal, point taken, OK.

Yes, let’s all have a nice cup of tea or a coke or a beer or a cocktail now. 🙂

Also from three days ago:

So Sal, do you see what eigenstate and keiths were getting at now?

Do you get my point to Andre above:

If so, maybe an edit to your post might be in order?

I confess, I do not think eigenstate is making much sense. His phrasing is idiosyncratic.

Nevertheless, out of my great respect for you, I edited the OP to include your comment as an ADDENDUM.

OK, thanks, Sal 🙂

So, which is it? Is what I said at #1 ridiculous, or is it “valid and easily demonstrated”?

tbh I think it’s true both ways.

I wish it wasn’t.

It’s much more interesting to try to find out why people think they are right, than why they are wrong.

Sal,

Actually, eigenstate’s phrasing is quite precise:

Every specific outcome has the same probability: 1 in 2^500. Every specific outcome — including all heads — is consistent with the physics of fair coins.

Your OP claims that

The OP is wrong. The decent thing to do is to acknowledge that and correct your error.

I can tell you what your are still saying which I think is wrong:

Statistically inconsistent. If you mean unlikely or improbable or highly improbable then say that.

You used ‘inconsistent’ in your original post that I replied to and you’re still using it now. Inconsistent does not mean the same as improbable or unlikely so . . . which is it you mean to say?

All coins being heads is only ‘maximally deviant’ from the expected value of the number of heads expected. It is NOT maximally different from other measures. For example: what if the measure was Kolmogorov information? What is the distribution of the measures of Kolmogorov information for all the possible outcomes generated by fipping a coint 500 times? Or what if the measure was length of the longest sequence of hesds in the 500 flips? Or what if the measure was the largest number of HH pairs in the sequence?

You might think that’s all just bunk and silly but you should look at the ways random number generators are evaluated. There are lots and lots of ways of ‘measurng’ a sequence of 500 tips of a fair coin. Why do you think the number of heads is the pertinent measure?

I was even more precise by saying it was 22-sigma from expectation, but that doesn’t seem to phase some of the die hards in this discussion.

If we go to even larger number that 500, we could have 100-sigma events, and yet some die hards will remain obstinate…

I’ve gone through a lot of trouble to be exacting in my descriptions. Neither you, nor Neal have refuted the problem of a phenomenon lying 22-sigma away from expectation. When a phenomenon is 22-sigma from expectation, that means something in standard practice…apparently the meaning of this isn’t appreciated by some in this discussion.

Did you see the graph illustrating the law of large numbers in action for dice? Can you face empirical illustrations like that and still insist all coins heads is consistent with probabilistic expectation? If you can, well, we’ll never agree on a lot of things…

You and Neal are fighting against the law of large numbers…

A resolution of the ‘all-heads paradox’Methinks keiths doesn’t understand the word “resolution”…

You can measure the number of tails too.

The measures, like deviation and expected value come from statistics that seem to work very well in the real world. Why it works might be a philosophical question, but to pragmatists like myself, I’ll keep using it as long as it works.

Sal, it’s not a philosphical question, it just a simple mathematical one.

There are simply more ways of arranging equal ratios of Heads and Tails than there are of unequal ratios.

Therefore there are more possible sequences with more equal ratios than with less equal ratios.

Therefore sequences with more extreme ratios will turn up more rarely than sequences with less extreme ratios.

And if something occurs less frequently, it’s less probable that you’ll see it the next time you throw.

It’s no more mysterious than that. Which is why all this stuff about 500 heads being against the Laws of Physics, or against the Law of Large Numbers is irrelevant. It’s not. It’s just rare. Much much rarer than a dodgy coin or a guy who laid them all out by hand, and so if we see one, the dodgy coin or the guy who laid them all out by hand is by far the most likely explanation. A near certainty, in fact.

(similarly, there are more fancy sequences than non-fancy, but the same principle applies).

Is this the same eigenstate that insisted it was theoretically possible for the moon to exist and not exist at the same time? Poor chap.

pardon, but wake me up when ID critics tell me they are concerned seriously that all the O2 molecules in the room they are sitting in, will all spontaneously rush to one end and manage to stay there for five minutes, leaving them gasping fruitlessly. (That is a comparable fluctuation to what they seem to want to seriously argue for and suffers the same threshold of observability problem.) KF