What "100 year flood" really means

How likely is a "Hundred Year Flood" this year? Does the likelihood change when you've just had one?

I have a subscription to an online new source. Many of the stories it publishes are open for comment. One of the reports was about a Turkish geologist, Naci Gorur, who was trying to raise earthquake awareness. I saved the following comment because it makes a crucial point about what the probabilities of "rare" events actually mean. The highlighted sentence sums up the math. Percentage odds are not intuitive. I've added the calculation below Repetto's comment. I used my computer's calculator to do the arithmetic.

[ by R.C. Repetto, Amherst, MA]

People can't deal with probabilities, such as "a hundred-year flood". If there was one ten years ago, they think they're safe for another 90 years. No, they face a one percent probability there will be one next year and more than a ten percent chance* there will be one in the next decade. That misunderstanding and shortsightedness is why people still move into disastrous locales, such as Florida or Phoenix or the mountainous regions of the West. It makes a mockery of the claim that "we" can adapt to climate change. We haven't and won't, until it's too late.

* If the odds of some event is 1 percent (one per hundred) per year, then the odds that it will happen within the next 10 years are (1.01^10*100)-100, or 10.4%

Footnote: If you knew there was a one percent chance of having an accident every time you drove your car, would you drive it?

How to Misunderstand Physics

 On Metaphor and Misunderstanding
Why physics is misunderstood

Originally part of  Usenet post Re: Empirical Utility of Dualism Posted: Dec 2, 2005 11:01 PM . Wolf Kirchmeir said: [...]
The three types of quarks could've been called anything at all. The terminology was preceded by the mathematical models that confirmed and predicted observations. The theoreticians could have used Greek letters, like they did for the tau, the mu, etc. Or Egyptian letters (which IIRC was actually suggested.)

Hint: learn the math.

"Quark" is borrowed from Joyce's Finnegan's Wake. Joyce borrowed the word from the German, wherein it refers to a kind of cottage cheese.

Me, I'd've proposed "flush" and "skint"; "womble" and "gronk"; and "tvepji" and "bsanji".

But nobody asked me. :-(

[A response to this post implied that naming the flavours of quarks up/down, top/bottom would lead to a “more interesting understanding” than the terms I suggested. “Flavours” is of course another metaphor. My comment on that post follows:]

The terminology was chosen to be deliberately arbitrary. The intent was to avoid what was called "a more interesting understanding," since quarks of all three types simply aren't like anything we can understand. Only the mathematical models make true sense of the phenomena they refer to. Ordinary-language accounts are metaphors, and like all metaphors they obscure as much as they illuminate.

It's somewhat like reading music. Some people can "hear the music" when they read the score, others (like me) can more or less accurately sing or play it, but for many a written score is just so many black spots, and they can't even "follow the score" when they hear the music played. When it comes to the mathematics of sub-atomic physics, very few of us can even follow the score, let alone pick out the tune or hear the music just by looking at the score. The physicists, bless their hearts, try to make their theories understood, but what their well-intentioned attempts actually do is foster a great deal of misunderstanding.

Addendum 2015-06-02: I think the misunderstanding applies to the physicists, too. I don’t think it’s useful to say that photons are waves or particles. All we know is that in some situations, we can use wave equations to describe their behaviour, and in other situations we can use particle equations. To say that the “wave function collapses” I think merely means that the probabilities described by the wave function are replaced by certainties when we observe/measure the consequences of some interaction. To refer to entities that interact as some entities that exist in and of themselves apart from the interactions is I think a mistake. All we can ever know is the interactions.

The Improbability Pirnciple: Why we don't notice the improbability of eveyday life (re-read)

 

David Hand. The Improbability Principle (2014) Suppose you’re playing bridge. You get a hand of all 13 hearts. How unusual! In fact, this deal is one of  635 013 559 600 possible hands. “Ordinary” hands are much more likely, right? Well, yes and no. The fact is that any combination of 13 cards is equally likely. The all-hearts hand is unusual only in that you notice it. A hand with a mix of values and suits looks normal, and it is, in the sense that there are only four all-suit hands, and 635 013 559 596 mixed-suit hands. But each one is unique. So each one is as unlikely to be dealt as any other. All bridge hands are equally improbable. 

     The same goes for lottery number picks.  

And when you have absorbed that fact, you are on the way to understanding Hand’s book. He explores odds and chance, our perceptions of odds and chance, and the tools available for estimating odds and chance more accurately. The exploration shows that “Coincidences, miracles, and rare events [will] happen every day”. He demonstrates several laws of probability that combine to make the improbable happen.

                                                  Ian Fleming was wrong.

    Hand’s book will help the reader realise how improbable every event is. It’s a good introduction to probability and statistics, with many real-life examples as well the standard text-book ones. It will help the reader see the world in which they live with more understanding, and I hope more curiosity. Hand writes well, his tone is conversational, he allows himself the occasional dry joke.
     Recommended. ****
     

Here’s my take on his work. It builds on his book, and other books I have read.
     Improbable events must happen, for there are long and convoluted chains of cause and effect leading up to every event. Call them event-chains. Looking forward from here and now, an enormous number of possible event-chains stretches into the future. They intersect and criss-cross in unpredictable ways. The future is a network of possible events. Any one event lies on a node, where several possible paths through the network meet. Which paths through this network could lead to events involving you, tomorrow morning, while you are having breakfast? An enormous number. You can list some of the most likely events (the cat will want to go out just before you set the breakfast table, you will fetch the cereal from the pantry, etc).

Jung didn't understand probability. Nor did he notice that "meaningful coincidence" is meaningless. What's meaningful for one person is a mere oddity to someone else, and triviality to a third.

      But there are other ones, trillions of them in fact (a meteor will crash into the garden, a storm will strip the leaves from the oak tree, two cars will collide in front of your house, the water heater will spring a leak, etc). The odds that any one of them will happen is small (the microwave will stop functioning). For most of them, the odds are very small (one of the people in the collision is a schoolmate whom you haven’t seen in twenty years). Some are extremely small (on the back seat of the blue car there’s a paperback that you donated to the Goodwill in another town seven years ago).
     One of these unlikely events will happen. True, some event-chains are more likely than others, but in general, there are far more unlikely possible events than likely ones. There are so many that unlikely events are more likely to happen than likely ones. The likely ones just happen more often.

      As with the all-hearts hand, most events are equally unlikely. Or equally unlikely enough that it makes no difference. We pay attention to the ones that we feel are strange in some way. (That's why Jung was wrong about "synchronicity".)

     Think about it this way:
     You go to buy a box of ball-point pens. Consider the event-chain leading up to your purchase. Dozens, perhaps hundreds, of people were involved in producing the raw materials, shaping them into parts, assembling them into pens, packaging them, distributing the pens to the store. Then there’s the event-chain leading up to your decision to buy the pens. Today, not yesterday. This store, and not another. And so on. What are the odds that you would buy this particular box of pens, today?
     Exactly.
     So why don’t you think of it as improbable?
     We don’t usually notice the improbability of any given event. That’s why we’re flummoxed when we do notice one.

 Another review of this book:  https://kirkwood40.blogspot.com/2016/05/the-odds-that-odd-things-will-happen.html

 

Blood tests and illness: do the arithmetic.

(Thanks to John Paulos for inspiration)

Suppose a nasty but relatively rare disease that shows up every spring. Suppose that in any given year, 5% of the population will get it, and many of them will die. Suppose that researchers have discovered that if you catch the virus early enough, a short but expensive treatment will cure you. Would be nice to have a simple and cheap blood test, wouldn’t it?

Suppose now you read a news story that a lab has developed a blood test that will find evidence of the disease before you have any serious symptoms. It’s cheap enough to use as a screening test every spring. Suppose it is 95% accurate. That means, it will catch 95% of the people who have it.

Sounds pretty good, right? 

Think again.

Test 1,000 people for the disease. 5%, that is 50 people, will have it. You will find 45 of them.

What about the 950 that don’t have it? At a 95% accuracy rate, 95% of those will test negative and 5% will test positive. 5% of 950 is 47.5. So 47 or 48 people will test positive that don’t have it. Let’s go with 47.

So after 1,000 people are tested, we have:

5 false negatives
903 true negatives
Ratio of false to true negatives: 5 to 903, or 1 to 180.6, or 0.006%, or very low.
If you test negative, the odds are close to 200 to one that you don’t have it. Pretty good.

45 true positives
47 false positives
Ratio of true to false positives: 45 to 47, or 0.95 to 1, or 96%, or almost even.
If you test positive, the odds are almost even that you do not have it. Leaves you pretty much where you were before the test.

So if you take the test, and it comes up positive, you have a roughly 50% chance that you don’t have it. If it comes up negative, you have a roughly 99% chance that you don’t have it.

You realise that a vaccine would be better.

Read the news with a numerically critical eye.

A Short History of Infinity

 

    Brian Clegg. A Short History of Infinity (2003) The title describes the book, and Clegg does a good job of introducing the “interested reader” to the mathematical concepts of infinity, and the many mathematicians who contributed to and developed the modern concepts of countable and uncountable infinities (terms he doesn’t use).
      The basic ideas are in fact simple: two sets are the same size (cardinality) if you can match them element for element with no exceptions and none left over. Using this rule you discover that the set of whole numbers is the same size as the set of  square numbers, or any other set whose elements can be defined in terms of some operation on the whole numbers.
      The trouble starts when you are stuck at the stage of thinking of numbers as somehow real, just as kittens and trees are real. Not a problem with the whole numbers or the rational fractions: you see five pies on the kitchen counter, each cut into six pieces, so you 30 pieces of pie, or 30/6ths of a pie. But in the 15th  and 16th centuries mathematicians began to work with numbers that you couldn’t point to in this way. We need and use “imaginary” numbers because they work, they enable us to solve problems in both pure and applied math that we couldn’t deal with otherwise. Clegg needs us to accept the weirdness of infinite numbers, so he spends several chapters on imaginary numbers.
     The first of these was negative numbers. Here Clegg  (who is above all a compiler of information) seems to have missed something: a negative number can be pointed to: if you have -5 dollars, then you owe $5 to someone. Negative numbers may have been discovered as points on the number line, enabling solutions to otherwise insoluble equations, but bookkeepers made them real.
     The square root of -1 (i) was the next “imaginary” number to be discovered (or invented:  the verb you use reveals your metaphysics). Clegg makes a big thing of this one, too, but he asks us to accept on faith his assurance that it is used every day by engineers. A couple of examples would have been helpful.
     Finally, he comes to Cantor, whose mental health was fragile, and whose feud with Kronecker (his erstwhile mentor and sponsor) triggered the final crisis. Cantor applied the axioms of set theory to infinite sets, and in doing so showed that “infinity” was a viable mathematical concept. In particular, it helped clarify the differences between rational, irrational, and transcendental numbers.
     A good book, its flaws are minor and don’t interfere with understanding infinity. Clegg likes explaining things, and has a neat talent for potted biographies that give us both the facts essential to understanding the subject’s place in the central story, but also enough quirks to make the people real. What you make of other questions about infinity (such as whether the Universe is infinite or not) is left up to you. Recommended. ***

Charles Seife. Zero: The Biography of a Dangerous Idea (2000)

     Charles Seife. Zero: The Biography of a Dangerous Idea (2000) It looks like I read this last year, but I can’t remember. Re-reading it, it’s not hard to see why. While Seife provides lots of interesting information, and explains a good deal, his writing is at best workmanlike, and often sloppy. I suspect the sloppiness may be the effect of trying to “explain complex ideas in simple language,” but the result is often conceptual blurriness and even error. I found myself mentally rephrasing many of his statements. For example. He says that Cantor compared the size of rational and irrational numbers. He of course means the size of the sets of those numbers. Since he already explained what a set is, and uses a very clear metaphor to illustrate how one compares sets without actually counting, there’s no excuse for such sloppy language. He does have a knack for the illustrative metaphor: comparing sets is like asking everyone in a stadium to sit down, he says. If some seats are empty, then there are fewer people than seats. If there some people left standing, then vice versa. If there are no empty seats and no people standing, then the two sets match: they are the same size. Well, that’s very well done; so why the sloppy language a page or so later?
     Seife also occasionally uses a technical term without explaining it. For example, towards the end he talks of the heat death of the universe as the ultimate result of its continuing expansion. But in the next sentence he refers to this as death by ice (in contrast to the fiery death of the big crunch). How these two terminologies can be reconciled may be a mystery to him; it certainly will be a mystery to many readers.
     Seife’s understanding of history consists of conventional wisdom, which also occasionally misleads the reader. Overall, however, his philosophical points are well made, and the power of Zero to confound metaphysics and theology is clearly conveyed. The appendices illustrating several mathematical and logical arguments in detail are concise and clear. I like the one that uses the a=b, ab=a^2 etc proof that 1=0 (or 2=1) to show that Winston Churchill is a carrot.
     In short, this is an adequate introduction to a number of mathematical, physical, and philosophical problems and their solutions, with a good deal of pleasantly conveyed history along the way, and will do for a high school library. ** (2002)

Philip J. Davis. The Thread: A Mathematical Yarn (1989)

     Philip J. Davis The Thread: A Mathematical Yarn (1989) A charming book, telling how the author, a mathematician, became curious about Pafnuty, the first name of his hero, Pafnuty Lvovitch Tschebyscheff, a pioneer in the mathematics of approximation. Approximation has become a central motif of computing, since every computer can calculate only to some finite number of decimal places. It was the rounding off the 17th digit to display a 16-digit result that led to the discovery of chaos theory. That tiny difference of a few parts in 100 quadrillion made all the difference when the result was fed back into the equations for a second run of a weather prediction model.
     But I digress. Which is what Davis does. Some of his digressions are personal, some technical, some historical. But he leads us down these byways so gracefully that we hardly notice that we are moving further and further away from the ostensible theme of the book: whare does the name Pafnuty come from? Davis brings the thread of his narrative back to this question several times, and finally gives us the answer: it derives from an Egyptian god’s name.
     Along the way, Davis instructs us in all manner of interesting facts. He illustrates one of my dicta: There is no such thing as useless knowledge; at the very least, a fact will serve to link two others. I’ll now add another corollary: and usually, this linkage satisfies our thirst for order and meaning. For order and meaning are fancy words for linkages.
     This is the second time I've read the book, and I enjoyed it just as much as the first time. ****

Impossibility (book)

John D. Barrow Impossibility (1998) There are several kinds of impossibility, but they fall into three groups. There is the practical impossibility, reflecting some limits to the resources we (or any other creature) can command. Then the nature of the Universe itself sets limits on the possible. And all logical systems above a certain level of complexity exhibit impossibilities.
     An example of practical impossibility is the solution of problems that would take more computing time than the lifetime of the Universe; another is travelling beyond the solar system. Whether the Universe has a beginning or not is an example of a question we cannot answer because, although we can specify what we should need to know in order to settle the question, we cannot get the necessary knowledge. An example of a logical impossibility is expressed in Godel's theorem, which states that any axiomatic system at least as complex as arithmetic contains statements whose truth or falsehood cannot be determined
     A more interesting example is Arrow's Impossibility theorem: as the number of candidates for office increases, the probability that there will be no majority winner. approaches certainty. What this means in practice is that whoever wins, most people wanted someone else. The result can be generalised to any situation with multiple, mutually independent choices . It also applies to sporting events. Where several teams compete for a championship, there is surprisingly large possibility that the winner can be (and often has been) beaten by one or more of the losers. With 8 teams, the odds of this happening are 1 in 3.
     Barrow is a somewhat turgid writer. There are irritating typographical errors throughout the book, mostly of the wrong-word variety; an effect of reliance on spell checkers. The book is heavy going in places. I have read similar discussions elsewhere, and so didn't get hopelessly lost, but anyone who hasn't at least a senior high school understanding of physics, logic, mathematics, and other disciplines will probably have trouble following some of Barrow's arguments. Nevertheless, it's worth reading, if only to disabuse one of the notion that all things are possible. Barrow's most subtle point is this: that impossibilities, the limits of action and knowledge, tell us more about the nature of our Universe than the possibilities do. *** (1999)

Update 2012: if quantum computers do become a reality, then the range of solvable problems will enlarge by many orders of magnitude. Then question then become which of these problems are worth solving, which may be impossible to answer without solving the problem.

Update 2019: Minor correcctions in style and spelling.

Mathematics and Plausible Reasoning. Vol. II: Patterns of Plausible Inference

George Polya Mathematics and Plausible Reasoning Vol. II: Patterns of Plausible Inference. (1954). I skipped Vol. I, which deals with mathematical induction. The two books are intended as texts, either for self-study or for a course. This purpose of this volume can be seen in this example. Given A -> B and B, what can be deduced? By formal logic, nothing; i.e., the truth of A cannot be inferred from the truth of the consequence. (However, if B is false, then A is false.) Polya shows that in fact the truth of A will be more or less credible depending on a number of relevant factors. For example, if the truth of B is less credible without the truth of A than with it, then B supports A. Or, if B is more credible, then A is more credible; and so on. IOW, the truth of A lies somewhere between 0 (false) and 1 (true).
     Polya notes that credibility of A depends in part on the judge's experience and background. He is very close to fuzzy logic here, but he doesn't take the next step because he can't see any supportable way to compute that value. Fuzzy logic formalises that personal judgement, and so can provide computations (which are used to control machines, e.g.) Polya uses probability theory, interpreting probability as credibility, and thus provides strong support for his POV. He's also interested in the use of plausible reasoning in mathematical research. An interesting book. I like its assumption that its subject is worth pursuing. Polya writes very clearly, and I was able to follow about half of the math. The general principles of plausible reasoning seem to me to be obvious. *** (1998) Update 2012-12: It seems to me that Polya was a pioneer of what became fuzzy logic, but I can’t recall any acknowledgement of this in the fuzzy logic text I read.

Flatterland (Book Review)

Ian Stewart Flatterland (2001) Ian Stewart is one of my heroes. He can make complex ideas simple enough that anyone with high school math should be able to understand them. He’s also something of a polymath: he ranges outside mathematics into biology and cosmology. See
http://en.wikipedia.org/wiki/Ian_Stewart_%28mathematician%29
for more, including a bibliography.
Flatterland book is a sequel of Abbott’s Flatland, a classic that has never been out of print.  See http://en.wikipedia.org/wiki/Flatland
     The book ranges far beyond geometry, however, leading up to the question “What shape is our universe?” Stewart brings Victoria Line (and us) to the answer step by step, with the guidance of the Space Hopper, who is able to take Vikki outside space. The mathematician’s need to avoid fuzzy thinking is demonstrated, which makes this an excellent book for anyone who wants to get past the notion that arithmetic is all there is to math. But what makes the book charming is Vikki’s character (she’s smart and courageous), and Stewart’s penchant for puns. There are frequent allusions and pastiches of the Alice books, too.
     I’m not sure how a mathematical naif would read this book. I find it difficult to say how much my prior knowledge of most of the concepts in this book helped me to follow the story. It certainly helped to get most of the jokes. However, I’ll recommend this book anyway. If you don’t get it the first time, read it again. The chapters are short, the illustrations not only conceptually accurate but often amusing. This is a book that will appeal to people who like to take their learning in small doses. ***

Math: Pyramids, Prisms, and Infinity

   I like math, because you can formulate questions that you know can be answered, even if you haven't always enough knowledge to answer them yourself.
   For example:  Take a tetrahedron, a pyramid of three triangles on a triangular base. If all four triangles are the same, then it's a regular tetrahedron, the smallest of the five regular solids. Like the other solids, you can, within limits, change the proportions of its faces in some way. You can increase the height of the triangles that make up the sides of the pyramid. How much? As much as you like.
   Now here’s the question: What happens when the height is infinite? Well, that depends on how you define “infinite” in this context. If by “infinite height” you mean “without limit”, then the tetrahedron becomes a pyramid of infinite height. As the height of the triangular sides increases, the pyramid becomes more and more like a prism of triangular cross-section. That is, the edges become closer and closer to being parallel. We can say that the difference between three edges that converge on a point (the apex of the pyramid) and three edges that are parallel becomes smaller and smaller. This difference approaches zero. “At the limit” it is zero: then the pyramid has become a triangular prism.
   Does it make sense to talk about a limit here, when we are talking about a pyramid of infinite height? Yes, on the same grounds that the differential calculus uses the concept of a limit. But this question, and its answer, are beyond my ability to explicate or justify. The best I can do is to notice that stretching the pyramid towards an infinite height is the same as rotating each edge about a point (the corner) so that they become parallel.  So the pyramid “eventually” morphs into a prism. That “eventually” is “at the limit”, when the difference between converging and parallel edges has become zero. It corresponds to a pyramid of infinite height. This implies that prisms as we conceive of them (of finite height)  are sections of infinitely high pyramids.
   I don’t know whether the above line of thought is mathematically acceptable. Maybe I’m mixing two branches of math illegitimately. But it feels right. So I’ll state my conclusion as a theorem:
   “A finite prism of N sides is a section of an infinitely high pyramid of N sides.”