Links for 2011-09-01

Some basic tips on finding useful information, and the rudimentary statistical knowledge you need to make sense of it. Most applicable in the social and life sciences, but worth knowing for anybody.

"I'm pretty sure I've posted this before in one form or another, and I'm not sure what prompted me to do it again, but every once in a while I feel the urge to present some raw data about how our kids are doing in school. The charts below are taken directly from the most recent NAEP report card, generally viewed as the "gold standard" among measures of student achievement. Here are the results among eighth-graders over the past 20 years"

"So, I recently read An Introduction to Information Theory: Symbols, Signals and Noise.

It is a very nice popular introduction to Information Theory, a modern scientific pursuit to quantify information started by Claude Shannon in 1948.

This got me thinking. Increasingly, people try to hold conversations on Twitter, where posts are limited to 140 characters. Just how much information could you convey in 140 characters?

After some coding and investigation, I created this, an experimental twitter English compression algorithm capable of compressing around 140 words into 140 characters.

So, what's the story? Warning: It's a bit of a story, the juicy bits are at the end."

"[I]magine my surprise - nay, dismay - to discover that publishing's streets were not paved with gold, but stalked by the anxious, the gloomy, the suicidal. "Publishing's dead!" shouted men in sackcloth on Bloomsbury street corners. I had arrived at the party, but the coats were being handed out, the drink had dried up and the hostess had collapsed.

So I asked myself (somewhat desperately, positively naively): are things really that bad? What is the actual state of book publishing in Britain? Can writers really only look forward to a life of penury? Or should I stick my head in the sand, if only to deaden the sound of commissioning editors weeping into their lattes?"