I spend a lot of time thinking about the scientific method. I don't mean that thing you learned in high school, where you make an observation, form a hypothesis, design an experiment etc etc. That's certainly part of the scientific method, but the linear formula that freshmen are typically forced to memorize sucks the life and interest out of what it is that my colleagues and I do on a daily basis.
Source: The fantastic "How Science Works" from UC Berkeley (click image)
The process of doing science is messy and complicated, and most of the time it doesn't work. There are false starts, bad experiments, bad interpretations, wrong or incomplete controls and sometimes you just forget to add salt to your buffer. But science is the only path to knowledge where all of these problems, the flaws of human planning, perception and cognition, are baked into the process and corrected for. One of the most important parts of that corrective influence, and what's missing from the scientific method as conceived in high school, is the discussion and debate that comes from sharing the results of our experiments with others.
Modified from "Understanding Science" - click for source
For the last hundred years at least, this has largely meant publishing findings in a journal. At their inception, journals were just ways to aggregate and disseminate the letters that scientists already sent around to each other - in other words, journals increased science communication. Not anymore.
These days, most published science remains locked in silos controlled by publishers. Even wealthy institutions like Harvard are being crushed under the weight of subscription fees, and if you don't belong to an institution, you're likely to pay around $30 for a single article! Scientific journals are now impediments to the free flow of scientific information.
However, as with so many things in the internet era, the old model is breaking down and the entrenched interests are kicking and screaming to hold onto their supremacy. Unfortunately for all of us, these entrenched interests have an advantage that other disrupted industries do not.
Because the rise of journals occurred in tandem with the rise of professional, publicly funded science, the two are now inextricably linked, to the point where the publication of discoveries in journals is necessary to maintain a career in academic science. Job prospects, grants and promotions all depend on the quality and quantity of publications.
Institutional barriers prevent individual researchers from innovating with publishing strategy because doing anything outside the standard model would hurt their ability to be promoted and more importantly, to get the grants they need to do science. The good news is that some enterprising people are starting to build journals that work within the current model of science publishing, but aim to disrupt it from the inside. Enter PeerJ, a new online-only journal that published its first papers today.
The Open Access movement has made great strides, but there is still a long way to go before all academic content can be freely read, broadly distributed and openly re-used by anyone, regardless of geography, education or wealth. A long way, but it is a worthy trip to undertake.
And so this is why we believe so strongly in the mission of PeerJ. We want to be a catalyst for change within the system of academic communication. We want to publish science in an effective, efficient, rapid, innovative, respectful, professional and, above all ‘Open’ manner.
PeerJ has built a new open access journal, following the likes of the Public Library of Science (PLoS) and the much more recent journal eLife. However, where PLoS and eLife seem to be trying to build classic journals with an open access (OA) model, PeerJ seems to be trying to innovate on everything about the publishing process, from open peer review, to the integration of altmetrics, to the simple idea of publishing articles as they come in (like a blog) rather than in separate issues.
They're even planning to launch a pre-print server, which means that scientists will be able to upload their work to establish primacy (physicists have had something like this for a while), that can serve as an OA source, even for papers that eventually get published in a non-OA journal.
A little while ago, in response to a boycott of the publisher Elsevier, I wrote
We don’t need any academic journal’s services anymore. If you publish in any journal, you are making it easier for them to take action that harms academic institutions, so you shouldn’t[...]
[T]he truth is, journals add very little value to science, and impose huge monetary costs, as well as costs in terms of delayed publication and limited distribution[...]
In my idyllic world, every lab has their own blog, and publishes their results in real time, sharing them on a site like ResearchGate. Individual figures can be indexed on something like FigShare. Scientists can post their negative or confusing data and ask the entire world for help, or talk about their research plans and get critiqued. Meanwhile,altmetrics are being generated in real time to asses the validity of data, and scientists peer review on their own blogs or at some central location. The distribution of scientific knowledge returns to the model of the 19th century – free and openly distributed – but now also instantly and globally distributed at the same time.
My model of every scientist with their own blog may be too unwieldy, but PeerJ has built a journal that does almost everything I've been looking for, and are poised to show that this model can work and that it's better. I also wrote:
Science benefits when the flow of information is unrestricted and everyone benefits when scientific knowledge advances. Journals no longer assist in the distribution of knowledge, they only impede it, and no one benefits from this arrangement except the journals themselves. It’s time for something new.
PeerJ might be that something new. Here's hoping they prove me right.
- Log in to post comments
This piece would be more persuasive if the tone were less revolutionary and more additive. PeerJ strikes me as a good thing. I wish them well. But Elsevier and countless other "traditional" publishers are also a good thing. We don't have to choose. Diversity in all things: demography, politics, and publishing.
@ Joseph - Thanks for the comment. I had written a response that got eaten somehow, but Mike just argued essentially what I was going to argue.
Thanks for this, Kevin -- a good analysis. My reading is that it's quickly going to become impossible to justify spending $3000 on an APC at a legacy journal when PeerJ's superior facility are available at $99.
Joseph, it's unquestionable that Elsevier and other barrier-based publishers used to be good things. But those days are long over. For a decade or more, now, they have been a net negative for science. The Shirky Principle describes precisely what they are doing: "Institutions will try to preserve the problem to which they are the solution". We don't have the problem of limited distribution any more -- except when we allow "publishers" to impose it on us. That has always been unacceptable; now it's untenable, too.
[adding website into the body of the comment so you get the link juice - kb]
Mike - I hadn't heard of The Shirky Principal, but that's exactly what I've been trying to articulate re: publishers for a while now, thanks!
I totally agree with your view of the scientific method being too simple. The method sort of fixes students to view things in a box and prevents them from thinking outside of it. Going from high school to college was a huge adjustment for me. One had to change from being in that closed-minded state in high school to a more analytical and open mind in college. The lack of critical thinking is a big issue with middle and high schools across America. Well, that's another topic. But anyways, I feel like the price of research publishing is so discouraging to scientists these days, and is a huge deterrent from becoming a researcher. I feel like colleges are veering away from the strict format of the scientific method; in my college experience, I have taken three biology courses so far and they definitely don't follow the normal high school view of the method. It focuses on thinking critically and expanding upon the scientific method. I feel like in the near future, this 'idyllic' world you speak of may come to life, and eventually break the financial barrier of researchers.
Starting this blog, I was unsure where it was headed but I'm glad I continued on. From the beginning of the blog, I totally agree. The step by step method of the scientific method that is taught throughout a child's education, is completely making the entire process of science seem rigid and boring. Anyone who has ever even taken a physics or chemistry class knows they had that "Ah ha" moment when they did their first interesting lab and realized that science is not just about "Step 1...Step 2..." There is so much more put into the process. I remember myself always wondering in high school, "Then what?" So you follow this rigid step by step process to get to some answer, but then what do you do? As you said, it used to be that you published an article in a journal. That was always the cookie cutter answer to that question. But in reality, how many people sit down with a scientific journal in the coffee shop when they could just pull it up with the free wifi offered? Researching and experimenting to get new information in the science realm is about getting facts and then letting others that are interested know what you found. Scientific journals do not fulfill that duty anymore. As a former nursing student, we had a computer program that was run by Elsevier. In my mind, that was the best combination of the two options. As students, we had information available at our fingertips that we needed to know and it was all through a laptop. With the cyber world freely at our hands, it would be nice to think that we could take advantage of this, especially in the economic world that we live in, and use this powerful resource to truly get scientific information out to the public where it needs to be.
Jessica - your last line, and the beginning of the original post, imply that the primary role of the public is a receptive one, i.e., to accept the information that is delivered to them by professional scientists. I don't believe that the public will continue to value science unless they view it as being first a mental toolkit that everybody can use to generate new knowledge, even if only local and particularistic knowledge of a sort that would never be publishable in a journal. For this, most people need a straightforward and not too complicated approach - such as the despised "ask a question, design an experiment, collect data." A philosophy professor might equally feel that logic as taught to children is boring and oversimplified in comparison to his/her own elaborate work, but the latter is inaccessible to most people, while the former, well learned, is another useful mental tool for anyone. I am convinced that the simplified versions of these cognitive skillsets (and the related one of intelligent observation) can be taught to children either in a boring way or in a useful, applied, interesting way. Most traditional schools can manage to make any subject boring, so science in particular is not the problem here.
@ Jane - You're point is well taken. I'll have to think on it some more, but I'm not sure if students, even young children, are well served with the simple approach. At the "understanding science" site that I linked to, they have different versions of that flow chart that are intended for different age groups.
I think that collaboration is key to any scientific method though. Because of our flaws, it would be easy to try to follow the simplified scientific method and convince yourself of a whole host of things that aren't true. Though perhaps this nuance is too complicated for a 7 year old.
I could not possibly agree with you more on your first point though, that science needs to be taught as something that is accessible to everyone, not just the people in the white lab coats.
Although your comments on the scientific method as taught to young students were no more than a supporting part of your argument, it's inspired a disproportionate response because, well, it's actually quite important.
My comment would be that this oversimplification is less worrisome for how it may discourage young students, than it is for how it affects the students who become scientists. That is to say, while all working scientists know as a matter of extensive experience that this isn't how they actually do science, their conceptual understanding of what science is and how it works is nevertheless built upon and limited by this schoolchild presentation.
For example, as Kevin writes, it encourages scientists to undervalue collaboration. More importantly and less obviously, it ultimately is the root cause of the vast undervaluation of replication. That's just one example where there's a cultural naive overestimation in the truth value of consensus scientific opinion that results from a naive assumption that the science that's done is done by definition in a way that is uniformly rigorous and rigorous in a way that is fully specified. There's little value seen in replication because the reliability of peer-reviewed experimental results is overestimated.
And from the other direction, there is the pervasive popular conception of science as primarily the work of individual and revolutionary individual genius. Here, too, is the denial of the collaborative and institutional aspect of science that follows from the oversimplified schoolchild conception. This leads to crankism and, less visibly but no less importantly, a romanticized misunderstanding of science in young students that later results in drop-outs and burn-outs and, also, arguably, the misallocation of education relative to employment.
I profited greatly from reading Sir Karl Popper's "Conjecture and Refutation." Thomas Kuhn's "The Structure of Scientific Revolutions." is also thought provoking,.
@ Keith - All excellent points. I'm less worried about professional scientists being influenced by the simplified scimed, though. By the time I got to graduate school, my understanding of the scientific method was shaped far more by the labs that I had been in and the professional scientists around me.
I'll note though that the problems you mention: lack of collaboration and lack of repetition, are both fed by the current model of publication. If you share ideas too freely, you might get scooped and lose that paper. Replication isn't rewarded since it's much harder to get replication studies published.
@ Jim - I keep hearing about The Structure of Scientific Revolutions, I really need to get myself a copy and read it. Hadn't heard of the other one, but I'll definitely give it a look.
Back in the late 1960s, there was some well thought out criticism of evolutionary biology, which led to a good bit of soul searching. We asked ourselves the question, "Are we really doing science, or only telling Just So stories?" Popper was discovered, and various colleagues stated in print, "I am a Popperian." (In later years, I have heard claims of postPopperianism.) I don't think any other area of science has done this sort of soul searching. I strongly recommend familiarity with Popper.
Funding agencies LOVE collaboration, if they don't outright demand it. I've heard speculation that this is a way of appearing to fund a longer list of institutions even though the actual quantity of research money is stagnant. But when you actually put in an application that involves multiple disciplines, usually either you have the nightmare of trying to go through multiple panel reviews or you submit to the most relevant possible panel, who know zilch about two-thirds of the project. And then maybe you get told that Agency X will not fund anything that has any component relating to Subject Y that any moron outside the federal government can see is relevant to human/taxpayer welfare, so you're SOL.