Conference proceedings are citable, stop double-dipping

Over the last several years there has been a trend among conferences such as the Bio-Ontologies Special Interest Group meeting at ISMB and the Semantic Web applications and tools for life sciences (SWAT4LS) to invite article submissions for people that want to present at the conference and then to subsequently invite presenters to expand their article in an "official" publication in an associated journal.  Since the PDFs of the articles submitted to the conference usually end up online, as they should, this results in a situation where first reviewers and later readers are often confronted with two versions of essentially the same article - typically with the same title, author list, and often the same abstract.  This causes problems for reviewers as this kind of overlap with prior work (even from the same authors) would typically be grounds for rejection - yet because of this bizarre arrangement with the conference, reviewers are supposed to treat the original article as if it were a pre-print of the first, despite the fact that it is a citable entity on the Web and never referred to as a preprint anywhere.

Conference organizers, please stop this madness.  Here are three models that would be better.

1. Following the International Biocuration Conference model, invite submissions directly to the partner journal first and then choose presenters from the successful submissions and independently submitted abstracts.  No confusion.  One good, citable paper.  Probably higher quality conference submissions.
2. Put the articles submitted to the conference in a pre-print server such as arXiv or BioRxiv and continue with the concept of an expanded article in a journal.
3. Do what the computer science community does and recognize contributions to conference proceedings as citable articles and do away with the attempt to get an 'official' journal publication in addition to the conference citation.

non-anonymous peer review

I spent this afternoon acting as a voluntarily non-anonymous peer reviewer - its scary.  I ended up advocating rejection of the article I was reading and I have to say that Vince Smith(see end of linked post) was absolutely right that the act of signing your review "keeps you in check".  Knowing from the outset that your words are going to be linked to your name can really change what you have to say - it certainly makes you think about it for a while longer.  It is scary though - I hope that I managed to convey enough of my reasoning and suggestions for ways to improve the article that the authors don't despise me and attempt to ruin my life...  I also hope that the editors of the journal manage to acquire at least one additional reviewer for this manuscript - safety in numbers! Or perhaps the editors will strip my name from my comments?  Time will tell I guess.

getting attention

I sent the first complete draft of my dissertation to my committee last week.  Yay!  

At my last meeting with them they promised a quick response if I got it to them that day - even saying that they would reserve time specifically for reading it.  Unsurprisingly, most have not responded as they had promised - in fact most have not said anything.  Boo!  This means that either I will submit the thesis without having the benefit of their comments or I will miss my window for graduating this spring - double boo.

As many of you are no doubt aware, this is not an atypical situation.  Whether it is a 200 page thesis document, a 2 page report for the local hospital, or a technical article being prepared for publication, it can be very difficult to get essential feedback.  As Michael Nielsen has succinctly put it, the critical resource in science today is the attention of scientists.  My committee isn't ignoring me out of spite, they are just unbelievably overworked people with limited amounts of attention to meter out and I happen to be lower on their personal totem poles than their grant applications, their manuscripts, their other students, their husbands, their wives, (and probably their dogs and cats).  

The question this brings up is, when its critical that you receive some attention from a scientist or two, how do you go about getting it?  Lets make this more specific and ask, when you need some one to review one of your papers (prior-to, or rather-than a journal), how do you go about acquiring that needed attention?  

The only way that I have addressed this problem is by asking friends and family for help.  This works well (depending on your friends and family) up to a certain extent, but has some significant shortcomings:

1. they like you and don't want to make you feel bad - which may influence their assessment of your work
   
2. they may reside within the same information cocoon that you do - which means they may have little additional knowledge to contribute 
3. they eventually get tired of helping you out because they have their own problems to deal with

What other options are there? I suggest two that both revolve around markets.  The first market exists and the second is yet to be created.

$Market #1$

In discussing whether it would be worth my time to take a free scientific writing course offered at my institute, a professor who had taken the course really encouraged me to take it.  When I asked what she got out of it, she said that the most important thing that she learned was the value of working with a professional editor.  She now pays an editor to review every research paper and every grant that she submits.  I found that a little strange.  The most valuable product of a class is to learn that you need to pay some one to help you do what the class was trying to teach you?  Weird.  I would have dropped it there, but in another conversation with a very talented and well respected author, the same advice appeared.  To write at a professional level, getting professional help appears to be a vital component.

Now, as a student or a post-doc making lets just say not a lot of money, this advice is about as valuable as another suggestion that I love to hear from friends that actually have savings and real jobs (or rich parents), "oh, you should really try to buy a house, its such an important investment and now is such a great time to buy".  Great, thanks.  As soon as my scholarship check comes in I'll head out to the real estate agent...   Lacking funds to actually pay money to an editor, what could I possibly provide in exchange for some scientific attention?

Market #2

Well, according to some definitions, you might actually call me a scientist.  In fact, several journals have successfully taken my scientific attention from me (without any form of compensation) and handed it out to other scientists in the form of peer reviews.  Maybe I could claim greater control over this process?  Maybe there is a way to generate a market within which I could pay for attention when I needed it with my attention at other times.  I'm not referring to the perhaps more exciting 'collaboration markets' that Dr. Nielsen discusses, at least not yet, I'm simply referring to a market for the direct exchange of literary review in scientific contexts.  

Here is the essence of the deal; I will exchange my attention in reading and commenting on your paper in exchange for your attention on mine.  

Here are some of the additional complexities that might make an implementation of this idea interesting;

1. the chance to accumulate 'reviewer points' so that the system could go beyond barter and towards a more complete kind of market
2. the opportunity for anonymity for authors and reviewers to ensure that you can always say what you think you should say
3. the opportunity for the lack anonymity - for reviewers to be acknowledged in future iterations of the work that they review
4. the chance for participants in the system to establish levels of trust - some reviews really are more valuable than others and this should be recognized

I think something like this is vital.  It opens up a wide range of new opportunities for improving the way science works.  Papers could be 'published' within this system and gradually accumulate findability-enhancing credibility (and improvements).  Such assessments of credibility could be used to form a continuous rating scale for 'publications' that would replace the unnecessarily binary nature of journal-based publishing without losing the filtering effect touted by its proponents.

Such a system might improve on preprint archives like Nature Precedings by both providing a direct incentive for scientists to contribute comments on papers (most papers are never commented on at all at the moment) and providing a very direct approach to the filtering problem.  

If anyone is interested in creating something along these lines, let me know.  I hope that I will be needing a job soon.

p.s.  Thanks to Mikele Pasin for thoughts we shared on the 'Paper Demolisher' at KCAP 2007 that are directly related to this post.