I sent the first complete draft of my dissertation to my committee last week. Yay!
At my last meeting with them they promised a quick response if I got it to them that day - even saying that they would reserve time specifically for reading it. Unsurprisingly, most have not responded as they had promised - in fact most have not said anything. Boo! This means that either I will submit the thesis without having the benefit of their comments or I will miss my window for graduating this spring - double boo.
As many of you are no doubt aware, this is not an atypical situation. Whether it is a 200 page thesis document, a 2 page report for the local hospital, or a technical article being prepared for publication, it can be very difficult to get essential feedback. As Michael Nielsen has succinctly put it, the critical resource in science today is the attention of scientists. My committee isn't ignoring me out of spite, they are just unbelievably overworked people with limited amounts of attention to meter out and I happen to be lower on their personal totem poles than their grant applications, their manuscripts, their other students, their husbands, their wives, (and probably their dogs and cats).
The question this brings up is, when its critical that you receive some attention from a scientist or two, how do you go about getting it? Lets make this more specific and ask, when you need some one to review one of your papers (prior-to, or rather-than a journal), how do you go about acquiring that needed attention?
The only way that I have addressed this problem is by asking friends and family for help. This works well (depending on your friends and family) up to a certain extent, but has some significant shortcomings:
- they like you and don't want to make you feel bad - which may influence their assessment of your work
- they may reside within the same information cocoon that you do - which means they may have little additional knowledge to contribute
- they eventually get tired of helping you out because they have their own problems to deal with
In discussing whether it would be worth my time to take a free scientific writing course offered at my institute, a professor who had taken the course really encouraged me to take it. When I asked what she got out of it, she said that the most important thing that she learned was the value of working with a professional editor. She now pays an editor to review every research paper and every grant that she submits. I found that a little strange. The most valuable product of a class is to learn that you need to pay some one to help you do what the class was trying to teach you? Weird. I would have dropped it there, but in another conversation with a very talented and well respected author, the same advice appeared. To write at a professional level, getting professional help appears to be a vital component.
Now, as a student or a post-doc making lets just say not a lot of money, this advice is about as valuable as another suggestion that I love to hear from friends that actually have savings and real jobs (or rich parents), "oh, you should really try to buy a house, its such an important investment and now is such a great time to buy". Great, thanks. As soon as my scholarship check comes in I'll head out to the real estate agent... Lacking funds to actually pay money to an editor, what could I possibly provide in exchange for some scientific attention?
Well, according to some definitions, you might actually call me a scientist. In fact, several journals have successfully taken my scientific attention from me (without any form of compensation) and handed it out to other scientists in the form of peer reviews. Maybe I could claim greater control over this process? Maybe there is a way to generate a market within which I could pay for attention when I needed it with my attention at other times. I'm not referring to the perhaps more exciting 'collaboration markets' that Dr. Nielsen discusses, at least not yet, I'm simply referring to a market for the direct exchange of literary review in scientific contexts.
Here is the essence of the deal; I will exchange my attention in reading and commenting on your paper in exchange for your attention on mine.
Here are some of the additional complexities that might make an implementation of this idea interesting;
- the chance to accumulate 'reviewer points' so that the system could go beyond barter and towards a more complete kind of market
- the opportunity for anonymity for authors and reviewers to ensure that you can always say what you think you should say
- the opportunity for the lack anonymity - for reviewers to be acknowledged in future iterations of the work that they review
- the chance for participants in the system to establish levels of trust - some reviews really are more valuable than others and this should be recognized
I think something like this is vital. It opens up a wide range of new opportunities for improving the way science works. Papers could be 'published' within this system and gradually accumulate findability-enhancing credibility (and improvements). Such assessments of credibility could be used to form a continuous rating scale for 'publications' that would replace the unnecessarily binary nature of journal-based publishing without losing the filtering effect touted by its proponents.
Such a system might improve on preprint archives like Nature Precedings by both providing a direct incentive for scientists to contribute comments on papers (most papers are never commented on at all at the moment) and providing a very direct approach to the filtering problem.
If anyone is interested in creating something along these lines, let me know. I hope that I will be needing a job soon.
p.s. Thanks to Mikele Pasin for thoughts we shared on the 'Paper Demolisher' at KCAP 2007 that are directly related to this post.