Dummies don't know that they are dumb...

Over this break, I've done some reading and an excessive amount of blogging. I've also fallen prey to the sales at book stores; one "fun" book that I picked off of the $1.99 rack was Marc Abrams book: The Ig Noble Prizes: Annals of Improbable Research. The Ig Noble Prizes have made the mainstream news.

One of the articles they talked about was the one given in psychology in 2000:

PSYCHOLOGY David Dunning of Cornell University and Justin Kreuger of the University of Illinois, for their modest report, "Unskilled and Unaware of It: How Difficulties in Recognizing One's Own Incompetence Lead to Inflated Self-Assessments." [Published in the Journal of Personality and Social Psychology, vol. 77, no. 6, December 1999, pp. 1121-34.]

Basically, the article demonstrates that the more ignorant one is on something, the less that one recognizes their ignorance. This sure rings true for me in my experiences in the class room, both as a professor and as a student.
For example, some of the hardest people to teach are relatively untalented freshmen who did well (grade wise) in high school. And, as for me, the higher I went in mathematics, the more ignorant I felt. For example, when I waltzed into graduate school for the first time, I took a trip to the university co-op book store. I went to the mathematics section and was astonished that I understood the titles of at most 10% of the books on the shelf! It was to only get worse...I've heard many a graduate student lament "I used to be good at math; now I am a moron!"

And, I feel my dumbest when I am at a mathematics research conference!

Are my football predictions accurate?

First, I'll mention one problem that I am working on over this break. Consider the standard plane in two-space. Then in, say, the first quadrant, consider the "wedge" shaped region W bounded by a line through (0,0) of slope m, where m > 0, and the x-axis. Let X denote some infinite set of points in W which has (0,0) as a limit point. Clearly it is possible to find a single variable function f whose graph runs though an infinite subset of W such that

• f is continuous on the entire real line
• f is differentiable except for possibly at x = 0.

Here is how one can obtain such an f : Select from W some infinite subset {xi} with the following property: r(i+1)where {r(j)} is some strictly decreasing sequence which converges to 0. Next, construct an polygonal infinite arc by connecting the points xi to each other via line segments, and then setting f(0)=0, and then choosing a real number sequence ti where ti converges to 0 and then setting xi = (ti, f(ti)). Clearly such an f is continuous. To make f differentiable: let yi denote the midpoint of the segment connecting xi to x(i+1). Then do a three point spline smoothing process to obtain a smooth segment through y(i-1), xi, y(i) such that the smooth arc has derivative equal to the slope of the segment containing yi for each i. This modification makes f smooth, except for possibly at t = 0
So the question is: how does one make f differentiable from the right at t = 0 ; that is, I want {limit as dt goes to zero) of (f(0+dt)-0)/dt to exist. The type of thing I am worried about would be things like f(t) = tsin(1/t) which has a limit as t goes to zero, but cannot be made to be differentiable there. I have some rather cumbersome ideas to try out over the next 4-5 days.

Now, about football predictions: I made some bowl game predictions here:

http://blueollie.blogspot.com/2006/01/bowls-and-my-prediction-nds-season.html

and my record was 17-9 straight up (on correctly picking the winner) and 15-10-1 (on correctly picking the winner vs. the published point spread). Example: at the Insight Bowl, Arizona State was favored to be Rutgers by 11.5 points. I picked Arizona State to win (which they did) but I picked Rutgers to cover ( that is, either win or lose by no more than 11 points). The final score was Arizona State 45, Rutgers 40, so I won "straight up" and also won "against the spread". On the other hand, Michigan was an 11 point favorite against Nebraska in the Alamo Bowl. I picked Michigan to win, but Nebraska to cover. Nebraska won 32-28, so I missed the "straight up" pick, but won the "spread" pick.

So, the question obtains: am I a good at making football predictions? I define "good" as being "better than chance". In statistics, one can safely say that they are better than random chance if the probability of, say, a coin flip doing as well is less than or equal to 5%.

So I decided to do a hypothesis test:

Ho: my predictions are no better than a random coin flip

Ha: my predictions are better than a random coin flip.

I used the binomial distribution (26 bowl games, a coin flip has .5 probability of being correct each time, so how many times will a coin be right?). In the days of old, it was usual to use the normal approximation to the binomial or to use tables; but now-a-days there are handy calculators such as this one: http://www.stat.sc.edu/~west/applets/binomialdemo.html

So if X is the random variable that denotes the number of times a coinflip would be correct out of 26 trials, we see that P(X>= 17) = .0843 and P(X>=18) = .0378. Now we see that x = 15 for the spread betting and x = 17 for the straight up betting (note: my point spread picks were correct 15 times and P(X>=15) = .2786. ) So that tells us that in neither case (straight up or by the spread) gives convincing statistical evidence that I can do better than a coin flip, though I was "close" to being better than a coin flip with my "straight up" predictions.