Last weekend, when talking about the new SAT, I attributed the low quality of the essays reproduced in the New York Times to the fact that this is a test with vague questions and a short time limit. Dave Munger was a little skeptical in comments, and I remarked that:
Somebody ought to get a bunch of bloggers together, and give them the writing SAT under timed conditions, and see what they come up with.
Dave said he'd be up for it, as long as he didn't have to grade the resulting essays. That struck me as a big roadblock, as well, but in thinking about it a bit more, I don't know that it would be that big a problem (more below the fold).
Really, the question would be a qualitative one: on a subjective level, do bloggers writing in a timed-test situation produce better work than top-scoring high-school students? All you would need to do is generate a bunch of responses, and post them, and let people make up their own minds.
It strikes me that it ought to be technically possible to set this up. All you would really need is some sort of web-based form that would provide a text entry box and a strict time limit. That seems like something you ought to be able to get out of one of the umpty zillion quiz/ survey packages on the web. Suitable questions ought to be readily available in one of the many SAT prep books out there.
Once people enter their text, you just collect the answers (with associated names and URL's, or anonymously), and post them to a blog or web page for people to read.
So, two questions: 1) If one wanted to do this, how would one go about it? and 2) Would there be any interest in seeing this done? I realize it's kind of a lot of work for a nerd-off, and doesn't offer the glamor of math limericks...
- Log in to post comments
Pulling it off would be relatively easy, I think:
First, recruit a bunch of willing victi-- I mean, bloggers.
Second, state a time when the question(s) will be posted to a blog.
Third, set a time for their essays to be posted by the bloggers participating.
So, if the questions were to be posted on this blog at, say, 12:00, you could require that the participating bloggers have their essays posted on their blogs by 13:00.
I think I like Chad's proposal a little better -- it allows for more people to participate, since you don't have to name a specific time. Hell, I think it also might work to use the honor system. Put up the question in a specified place, then just ask writers to "be honest" about how long they took.
One other issue: really it's a bit unfair for bloggers to get to type their responses. Should we make them handwrite them and then scan them in?
I like the random time idea. You could do it three times, over three days, with different questions. And I think email would work better than blogs, since that way, you know when it's sent and when it's completed...or, you could write a simple program that demands people login to see the question, and only gives them an hour to complete it. (I have friends who could do this; host it too, probably, if you're interested.)
And the grading's a snap. I did holistic grading of these things for one unbearably long weekend, and I'm sure there are other humanities bloggers who've done the same. All we demand is links.
Lots and lots of links.
I grade SAT essays all the time for my SAT prep courses. I'll happily do the same here. I can even provide some SAT Essay questions.
The hardest part may be the timed portion. Not that SciBloggers wouldn't go by the honor system, but we want to be all scientific and such about this, yes?
Well *of course* a self-selected bunch of adults who regularly write short essays for fun (i.e., bloggers) are likely to be better at that task than a bunch of disinterested high school kids. So?
I was thinking of doing it as a timed web thing for more or less the reasons Dave mentions-- that way, you could have more people participate, and wouldn't need to coordinate it for a particular time.
I know there are some academic packages that let you do this sort of thing-- I could set it up on Blackboard for one of my classes, for example-- but I'm not sure how to do it in something that's generally accessible.
Well *of course* a self-selected bunch of adults who regularly write short essays for fun (i.e., bloggers) are likely to be better at that task than a bunch of disinterested high school kids. So?
My contention is that the quality difference might not be as great as you would think, because a lot of the better bloggers do a fair amount of revising that you wouldn't be able to do in a timed-test situation. I could be wrong about that-- Bora certainly seems to be able to bang out 1500 words in a remarkably short amount of time-- but I think there's a difference between blogging at leisure about a topic that happens to strike your interest, and writing a good essay in a strictly limited time about a topic that may or may not inspire you.
You've also got to take into account that those essays are handwritten. I know I type much faster when I write, and even in 25 minutes I could do a significant amount of editing. But then again, I was taking the SAT II Writing test (which is essentially the same as the writing on the new SAT) two years ago. And my high school put a huge emphasis on writing essays fast, so whipping out a 4/5 paragraph essay in 25 minutes was really nothing.
Use Survey Monkey (surveymonkey.com). The basic information collected includes a start and stop date-time, so you could easily check the elapsed time. The only problem is that there isn't a good limitation on the number of characters in the essay. However, this probably isn't a killer problem. Survey Monkey is very cheap to subscribe to (until you get into over 1,000 respondents/month).
Hey, I've got a survey monkey subscription -- I didn't even think of that. That should work fine. Chad, why don't you contact me via email if you want to try this out? Dave at wordmunger dot com.
You've also got to take into account that those essays are handwritten. I know I type much faster when I write, and even in 25 minutes I could do a significant amount of editing.
I haven't given that much thought, because it's probably a push for me. I don't type all that fast, and when I do type things, I do a lot of stop-and-start stuff, backspacing to fix mistakes and so on. The total rate of production probably isn't too much different.
Also, my handwriting is terrible.
We could always shorten the time limit a bit to account for the difference between typing and hand-writing.
Use Survey Monkey (surveymonkey.com). The basic information collected includes a start and stop date-time, so you could easily check the elapsed time. The only problem is that there isn't a good limitation on the number of characters in the essay.
I'm not sure there needs to be a limit on the number of characters. I guess the SAT thing is limited to two pages, but I'm not sure how much of a factor that is.
Okay, I've made a sample SAT writing test on Survey Monkey. You can try it out here:
Click here to take the essay test.
The instructions probably need to be revised to more thoroughly explain the online format, but I think it's not bad. I tested it, and it does indeed record the starting and finishing time. It would be better if it gave test-takers a clock, too. (I accidentally posted this on the other SAT post too -- sorry about that)
I'm in.