The third classroom experiment returns to an exercise that I posted about on 6 March, at the start of semester. At the end of an introductory quiz, I asked the class:
Looking around the classroom, what percentile of the relative distribution do you expect to end up? For example, 100 means you expect to top the class, 75 means you expect to outperform 75% of the class. 50 means you expect to be at the middle of the distribution, 25 means you expect to outperform 25% of the class. 1 means that you expect to be at the bottom of the distribution.
My March post dealt with the fact that there was a rather strong ‘Lake Wobegon Effect’ in the data, with no student saying that they expected to end up below the 50th percentile. Now that we have the final grades, we can ask the question: how well did students predict their relative rank? Below is the relationship between students’ predicted percentile in the distribution (horizontal axis), and their actual percentile (vertical axis). If all students correctly predicted their rank in the class, all the dots should line up along the red line. By contrast, the green line shows the fitted relationship. Students who rated themselves more highly did in fact do a smidgin better, but the relationship is very weak.
Now, here’s what happens when I break the sample into male and female students.
It seems that female students were substantially better at predicting their relative rank than male students. Does anyone know of psychological theories that are consistent with this?
Andrew Leigh's Archived Blog 2004-2010 
Andrew not sure about psychological theories but I do have a statistical explanation. The six students with the highest predicted percentiles are split into three males all of whom do much worse than they predict and three females who do much better relative to their three confident male counterparts. So in looking at the scatters the gender differences seem to be largely driven by these “outliers”. Take them away and you’re struck by the substantial variability in the data.
Two questions. Why not ask them their predicted grade rather than percentile? I suspect that would spread out the predictions and avoid the clustering at 50. Second did you get ethics approval for this and your other “experiments”?
Denzil, I think the grades idea is an excellent one – can I leave it to you to implement, and report back the results when my blog gets going again?
As to ethics, these are teaching tools rather than research (I reported most of the results back to the class as we went – including showing them the above plots with ‘grade thus far’ on the y-axis).
Not sufficiently psychologically trained, but the propensity for self-delusion matters on other fronts too. If you check out surveys like the NLSY where people are asked how many hours a week they work, a not insignificant percentage of men respond with the maximum possible hours (ie literally every hour of every day). Women are much less likely to do this.
The fact that no-one predicted they’d be below the 50th percentile is a worry. I agree that asking for grades (where they can give a low grade, but pretend that this doesn’t make them below average) would probably give a better look.
At TAFE and university in some courses, students are asked to grade themselves (I know someone completing BA Fine Art a few weeks ago that was asked to do this in several units).
The result, judging by your effort, are more likely to represent how much a student is willing to grab than how well they are doing. Self-serving bias is a biggie!
Good evening,
Freakonomics blog linked to this article, & I’m a nerd for behavioral studies so I came to check it out. As a recent graduate of college, I found the percentile question almost dated in what it was designed to ask. Points are used to determine grades, but a 96% is no better than a 94%… they are both A grades. Similarly, if there were eight B grades of varying point value, it wouldn’t matter to anyone – they are equivalent for all purposes.
If I was expecting to get an A in the class, regardless of how many other students got A grades, I might have marked myself in the 100%, even though statistically this is inaccurate. But the overall impression of the survey is interesting – no one in your class (about 35 students?) felt that they were below average compared to the others. Was this an honors class? I wonder what would happen if you asked each student to do the same survey but thinking about attractiveness (like Freako blog), intelligence, work ethic? In a big lecture hall (bigger sample) would this fit more accurately to the perfect fit?
I do enjoy a self-perception study, & I enjoyed reading your post. Denzil covered this well with his suggestion, & I don’t mean to beat a point down that wasn’t welcome. Cheers.
very interesting post.
In a study dealing with overconfidence of stock market investors Guiso and Japelli find that men are more prone to be overconfident than women.
(http://www.cepr.org/pubs/new-dps/dplist.asp?dpno=5901)