getting attention

I sent the first complete draft of my dissertation to my committee last week.  Yay!  

At my last meeting with them they promised a quick response if I got it to them that day - even saying that they would reserve time specifically for reading it.  Unsurprisingly, most have not responded as they had promised - in fact most have not said anything.  Boo!  This means that either I will submit the thesis without having the benefit of their comments or I will miss my window for graduating this spring - double boo.

As many of you are no doubt aware, this is not an atypical situation.  Whether it is a 200 page thesis document, a 2 page report for the local hospital, or a technical article being prepared for publication, it can be very difficult to get essential feedback.  As Michael Nielsen has succinctly put it, the critical resource in science today is the attention of scientists.  My committee isn't ignoring me out of spite, they are just unbelievably overworked people with limited amounts of attention to meter out and I happen to be lower on their personal totem poles than their grant applications, their manuscripts, their other students, their husbands, their wives, (and probably their dogs and cats).  

The question this brings up is, when its critical that you receive some attention from a scientist or two, how do you go about getting it?  Lets make this more specific and ask, when you need some one to review one of your papers (prior-to, or rather-than a journal), how do you go about acquiring that needed attention?  

The only way that I have addressed this problem is by asking friends and family for help.  This works well (depending on your friends and family) up to a certain extent, but has some significant shortcomings:

1. they like you and don't want to make you feel bad - which may influence their assessment of your work
   
2. they may reside within the same information cocoon that you do - which means they may have little additional knowledge to contribute 
3. they eventually get tired of helping you out because they have their own problems to deal with

What other options are there? I suggest two that both revolve around markets.  The first market exists and the second is yet to be created.

$Market #1$

In discussing whether it would be worth my time to take a free scientific writing course offered at my institute, a professor who had taken the course really encouraged me to take it.  When I asked what she got out of it, she said that the most important thing that she learned was the value of working with a professional editor.  She now pays an editor to review every research paper and every grant that she submits.  I found that a little strange.  The most valuable product of a class is to learn that you need to pay some one to help you do what the class was trying to teach you?  Weird.  I would have dropped it there, but in another conversation with a very talented and well respected author, the same advice appeared.  To write at a professional level, getting professional help appears to be a vital component.

Now, as a student or a post-doc making lets just say not a lot of money, this advice is about as valuable as another suggestion that I love to hear from friends that actually have savings and real jobs (or rich parents), "oh, you should really try to buy a house, its such an important investment and now is such a great time to buy".  Great, thanks.  As soon as my scholarship check comes in I'll head out to the real estate agent...   Lacking funds to actually pay money to an editor, what could I possibly provide in exchange for some scientific attention?

Market #2

Well, according to some definitions, you might actually call me a scientist.  In fact, several journals have successfully taken my scientific attention from me (without any form of compensation) and handed it out to other scientists in the form of peer reviews.  Maybe I could claim greater control over this process?  Maybe there is a way to generate a market within which I could pay for attention when I needed it with my attention at other times.  I'm not referring to the perhaps more exciting 'collaboration markets' that Dr. Nielsen discusses, at least not yet, I'm simply referring to a market for the direct exchange of literary review in scientific contexts.  

Here is the essence of the deal; I will exchange my attention in reading and commenting on your paper in exchange for your attention on mine.  

Here are some of the additional complexities that might make an implementation of this idea interesting;

1. the chance to accumulate 'reviewer points' so that the system could go beyond barter and towards a more complete kind of market
2. the opportunity for anonymity for authors and reviewers to ensure that you can always say what you think you should say
3. the opportunity for the lack anonymity - for reviewers to be acknowledged in future iterations of the work that they review
4. the chance for participants in the system to establish levels of trust - some reviews really are more valuable than others and this should be recognized

I think something like this is vital.  It opens up a wide range of new opportunities for improving the way science works.  Papers could be 'published' within this system and gradually accumulate findability-enhancing credibility (and improvements).  Such assessments of credibility could be used to form a continuous rating scale for 'publications' that would replace the unnecessarily binary nature of journal-based publishing without losing the filtering effect touted by its proponents.

Such a system might improve on preprint archives like Nature Precedings by both providing a direct incentive for scientists to contribute comments on papers (most papers are never commented on at all at the moment) and providing a very direct approach to the filtering problem.  

If anyone is interested in creating something along these lines, let me know.  I hope that I will be needing a job soon.

p.s.  Thanks to Mikele Pasin for thoughts we shared on the 'Paper Demolisher' at KCAP 2007 that are directly related to this post.

coverage of Pubmed by Citeulike

In the life science domain, the total number of items described by social tagging systems is currently tiny in comparison to the number of resources described by institutions. To illustrate, the MEDLINE bibliographic database contains over 16 million references while, as of November 9, 2008, Citeulike, the largest of the academic social tagging services, contained references to only about 203,314 documents with known PubMed identifiers.

Though important to see where things stand today, the interesting aspect of these new systems right now is their potential for growth. Given the large numbers of contributors (and very large numbers of potential contributors), it seems possible that their coverage might eventually meet or surpass that of resource-contrained institutional mechanisms. In 2007, the NLM reported that it indexed 670,943 citations for the MEDLINE database which equates, on average , to about 56,000 citations per month. To estimate if social tagging services might someday reach the same level of throughput as the NLM indexing service, we compared the rates of growth, per month, for MEDLINE and for Citeulike (on Pubmed citations) over the last several years and used this data to make some predictions of future trends.  Here is what we came up with.

The figure plots the numbers of distinct Pubmed citations described by users of Citeulike and by NLM indexers each month and, using exponential smoothing, plots an extrapolation of the observed trends several years into the future. Based on the data obtained so far, we find that the numbers of biomedical resources described per month by Citeulike users is increasing more rapidly than the the number indexed per month by MEDLINE and that, if current trends continue, Citeulike coverage would catch up with MEDLINE around the year 2014 - at which point both systems would be describing approximately 70,000 biomedical citations per month. As the rapidly expanding confidence intervals illustrate, there is insufficent data to provide strong evidence for the precise point of intersection or even that Citeulike will continue to grow; however, it seems plausible that Citeulike and other scientifically oriented social tagging services will continue to expand in their coverage of the life sciences domain at a faster rate than institutional systems and thus will eventually catch up to the point where every document indexed by a professional is also tagged for personal use by a scientist (or 10).

So, what are we going to do with all of that data ?

  

Mark's Favorite Poem

My advisor just resent this poem out to me and my laboratory.  Words for any scientist, engineer, artist or other creator-type to keep in mind and a poem that will always remind me of Dr. Mark and my time here in the city in the clouds.

The Calf-Path

by Sam Walter Foss (1858-1911)

One day, through the primeval wood,

A calf walked home, as good calves should;

But made a trail all bent askew,

A crooked trail, as all calves do.

Since then three hundred years have fled,

And, I infer, the calf is dead.

But still he left behind his trail,

And thereby hangs my moral tale.

The trail was taken up next day

By a lone dog that passed that way;

And then a wise bellwether sheep

Pursued the trail o’er vale and steep,

And drew the flock behind him, too,

As good bellwethers always do.

And from that day, o’er hill and glade,

Through those old woods a path was made,

And many men wound in and out,

And dodged and turned and bent about,

And uttered words of righteous wrath

Because ’twas such a crooked path;

But still they followed — do not laugh —

The first migrations of that calf,

And through this winding wood-way stalked

Because he wobbled when he walked.

This forest path became a lane,

That bent, and turned, and turned again.

This crooked lane became a road,

Where many a poor horse with his load

Toiled on beneath the burning sun,

And traveled some three miles in one.

And thus a century and a half

They trod the footsteps of that calf.

The years passed on in swiftness fleet.

The road became a village street,

And this, before men were aware,

A city’s crowded thoroughfare,

And soon the central street was this

Of a renowned metropolis;

And men two centuries and a half

Trod in the footsteps of that calf.

Each day a hundred thousand rout

Followed that zigzag calf about,

And o’er his crooked journey went

The traffic of a continent.

A hundred thousand men were led

By one calf near three centuries dead.

They follow still his crooked way,

And lose one hundred years a day,

For thus such reverence is lent

To well-established precedent.

A moral lesson this might teach

Were I ordained and called to preach;

For men are prone to go it blind

Along the calf-paths of the mind,

And work away from sun to sun

To do what other men have done.

They follow in the beaten track,

And out and in, and forth and back,

And still their devious course pursue,

To keep the path that others do.

They keep the path a sacred groove,

Along which all their lives they move;

But how the wise old wood-gods laugh,

Who saw the first primeval calf!

Ah, many things this tale might teach —

But I am not ordained to preach.