Skip to content
May 11, 2009 / steve8

the Quantitative GRE has a Quantitative failure:

The quantitative exam lumps a large portion of test takers with a significant variation in ability into a single score.
The verbal does not have a similar flaw.
the results from everyone who is trying to get into a graduate program:
allall_v
clearly there is a problem here where the most common score of all test-takers is the maximum.
let’s take a closer look at how this makes the GRE Quantitative even less useful for specific graduate programs:
quantitative: verbal:
   
cs cs_v
math math_v
ee ee_v
notice in the students pursuing a PhD in math, computer science or electrical engineering, the quantitative exam is too easy, too many students are binned together at the highest score simply cause the test is too easy for them. interestingly the above fields have 2 humps in verbal, i imagine the first hump is from non-native English speakers going into these fields of study, the 2nd hump is likely the native English speakers.
phil phil_v
With philosophy PhD students here we see a more desirable distribution, but still, the perfect is almost as frequent as the mode. A relatively normal distribution, only 1 hump and the perfect is far from being the mode.
hist hist
psych psych_v
md md_v
english english_v
in the 4 areas of study above, the quantitative GRE exam does a good job at differentiating the the people.  The distributions are very normal, with the max and min score occurring far less than the mode. Even in the fields of study which attract the most verbally talented people, the verbal GRE test doesn’t break down, it still yields relatively normal distributions where the mode is far more common than the max or min.
lets do a real basic analysis of what’s going on:
the actual distribution of quantitative ability probably looks something like this:
1
note: normal curves have no maximum or minimum value of ability, just like there is no maximum IQ value, its just insanely unlikely to have an IQ of , say , 500… but still possible.
With practical limitations like a finite time limit, I can understand the need for bounds on the scores.
When you set a maximum, everyone with an ability higher than the maximum shall now be bundled at that maximum value, so lets look at what that will look like with a max=800 and min=200:
2
here you see unnaturally high probability to achieve a 200 and an 800,  the combined probability to score higher than the maximum will be lumped into that one score.  This cannot be avoided, but the number of people who would naturally score higher than the max can be minimized as to not be too significant, like the verbal GRE does.
So Why is the Quantitative so much worse than the Verbal GRE??
well the Verbal GRE has a mean of 477, almost equidistant to the max and min, while the quantitative has a literal (but misleading) mean of 579, so the entire distribution is nudged closer to the maximum boundary,  its also a bit wider than the verbal (higher std dev)
3
see the massive data point at an 800 score?  its value is equal to the accumulated area under entire tail end of the curve (above 800).
which of course looks very similar to the actual GRE Quantitative distribution:
allso this is a fundamental issue with the test and should be corrected.
It can be fixed by lowering the mean to ~500 by making all questions more difficult across the board.
for those interested, the plots were done in maple of testdist2, where:
image
and
testdist2 := (min, max, x, mean, stdv) -> piecewise(x = min, int(distr(y, mean, stdv), y = -infinity .. min), min < x and x < max, int(distr(y, mean, stdv), y = 10*floor((1/10)*x) .. 10*floor((1/10)*x)+10), x = max, int(distr(y, mean, stdv), y = max .. infinity), 0) ;
using a plot command like this:
mean1 := 579; stdv1 := 130; gremax := 800; gremin := 200; plot(testdist2(gremin, gremax, x, mean1, stdv1), x = gremin .. gremax, numpoints = 100, style = point, symbol = cross, symbolsize = 8, labels = [ability, `~probability`], title = "Normal Curve with mean of 579, std deviation of 130, score increment = 10")
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: