I sent the first complete draft of my dissertation to my committee last week. Yay!
At my last meeting with them they promised a quick response if I got it to them that day - even saying that they would reserve time specifically for reading it. Unsurprisingly, most have not responded as they had promised - in fact most have not said anything. Boo! This means that either I will submit the thesis without having the benefit of their comments or I will miss my window for graduating this spring - double boo.
As many of you are no doubt aware, this is not an atypical situation. Whether it is a 200 page thesis document, a 2 page report for the local hospital, or a technical article being prepared for publication, it can be very difficult to get essential feedback. As Michael Nielsen has succinctly put it, the critical resource in science today is the attention of scientists. My committee isn't ignoring me out of spite, they are just unbelievably overworked people with limited amounts of attention to meter out and I happen to be lower on their personal totem poles than their grant applications, their manuscripts, their other students, their husbands, their wives, (and probably their dogs and cats).
The question this brings up is, when its critical that you receive some attention from a scientist or two, how do you go about getting it? Lets make this more specific and ask, when you need some one to review one of your papers (prior-to, or rather-than a journal), how do you go about acquiring that needed attention?
The only way that I have addressed this problem is by asking friends and family for help. This works well (depending on your friends and family) up to a certain extent, but has some significant shortcomings:
- they like you and don't want to make you feel bad - which may influence their assessment of your work
- they may reside within the same information cocoon that you do - which means they may have little additional knowledge to contribute
- they eventually get tired of helping you out because they have their own problems to deal with
$Market #1$
In discussing whether it would be worth my time to take a free scientific writing course offered at my institute, a professor who had taken the course really encouraged me to take it. When I asked what she got out of it, she said that the most important thing that she learned was the value of working with a professional editor. She now pays an editor to review every research paper and every grant that she submits. I found that a little strange. The most valuable product of a class is to learn that you need to pay some one to help you do what the class was trying to teach you? Weird. I would have dropped it there, but in another conversation with a very talented and well respected author, the same advice appeared. To write at a professional level, getting professional help appears to be a vital component.
Now, as a student or a post-doc making lets just say not a lot of money, this advice is about as valuable as another suggestion that I love to hear from friends that actually have savings and real jobs (or rich parents), "oh, you should really try to buy a house, its such an important investment and now is such a great time to buy". Great, thanks. As soon as my scholarship check comes in I'll head out to the real estate agent... Lacking funds to actually pay money to an editor, what could I possibly provide in exchange for some scientific attention?
Market #2
Well, according to some definitions, you might actually call me a scientist. In fact, several journals have successfully taken my scientific attention from me (without any form of compensation) and handed it out to other scientists in the form of peer reviews. Maybe I could claim greater control over this process? Maybe there is a way to generate a market within which I could pay for attention when I needed it with my attention at other times. I'm not referring to the perhaps more exciting 'collaboration markets' that Dr. Nielsen discusses, at least not yet, I'm simply referring to a market for the direct exchange of literary review in scientific contexts.
Here is the essence of the deal; I will exchange my attention in reading and commenting on your paper in exchange for your attention on mine.
Here are some of the additional complexities that might make an implementation of this idea interesting;
- the chance to accumulate 'reviewer points' so that the system could go beyond barter and towards a more complete kind of market
- the opportunity for anonymity for authors and reviewers to ensure that you can always say what you think you should say
- the opportunity for the lack anonymity - for reviewers to be acknowledged in future iterations of the work that they review
- the chance for participants in the system to establish levels of trust - some reviews really are more valuable than others and this should be recognized
I think something like this is vital. It opens up a wide range of new opportunities for improving the way science works. Papers could be 'published' within this system and gradually accumulate findability-enhancing credibility (and improvements). Such assessments of credibility could be used to form a continuous rating scale for 'publications' that would replace the unnecessarily binary nature of journal-based publishing without losing the filtering effect touted by its proponents.
Such a system might improve on preprint archives like Nature Precedings by both providing a direct incentive for scientists to contribute comments on papers (most papers are never commented on at all at the moment) and providing a very direct approach to the filtering problem.
If anyone is interested in creating something along these lines, let me know. I hope that I will be needing a job soon.
p.s. Thanks to Mikele Pasin for thoughts we shared on the 'Paper Demolisher' at KCAP 2007 that are directly related to this post.
Posts
8 comments:
Instead of market-focus and finding drives there, maybe we can start sending unresponsive professors on a guilt-trip so that they might improve their work ethics? Sure, most of them are indeed terribly overworked and students tend to end up lower in the priority list than the urgent deadlines, but there is something called responsibility that professors have: to educate the next generation of scientists.
If current professors go for their own short-term interests instead of giving constructive feedback to students at appropriate times so that they can learn and grow, then little will remain of science in 20-30 years time when the current batch of professors has retired. Many a time I've heard current professors lamenting the poor level of PhD theses and postdocs (the latter more and more in vogue because the PhD may not give enough academic baggage for conducting independent research, whereas in theory it is supposed to)---but maybe they should read first about the The Hidden Crisis in Graduate Education which concludes the research that "The source of graduate student attrition is not inadequate students but indifferent and wasteful programs". Well then, you reap what you sow, and if little is sown, there isn't much to reap.
There are several systemic issues that lead to such a situation, from the commercialisation of science, grant-based research where a lot of time is simply totally wasted, to the "Bologna process" to cut down on education years and funds (and, well, the academic culture selects for people with certain types of social skills). In the short run, it's people of our 'generation' of fresh PhD-ers that misses out, in the long run, it is society at large.
It would be interesting to do a survey among recent graduates to ask if their supervisor(s) actually had read the thesis just to see how prevalent it is (but check here, section 5, for an idea that it is indeed a recurring theme).
On your second item: maybe Nature precedings, or elsewhere, could add something along the line of the arxiv blog, where there are comments on articles posted on arXiv quite regularly.
Sounds like I hit a nerve there...
First, I guess I should clarify my situation a bit. I'm not complaining about my advisor (who did read and comment on my thesis in a timely manner and reads this blog occasionally (Hi Mark!), nor did I mean for this post to be yet another lament about graduate school. I'd really much rather talk about new kinds of systems for organizing intellectual labor like the scientific writing market I suggested, but since you brought up the education issue, I'll clarify my take on the problem that inspired this post.
In my program, students have one or two direct advisors who are responsible for making sure that the student has enough money to survive, has a reasonable environment to work in, and for guiding the student's research. PhD students also have a thesis committee. The thesis committee is composed of three or four faculty members whose responsibilities to the student include unprepared attendance at 1 meeting per year; that is all. The first meeting is generally devoted to introductions and possibly some discussion of roles and some preliminary planning. The second meeting is usually the comprehensive/qualifying exam - here, an oral examination that must be passed after all classes are finished and before the student is officially considered a PhD candidate and supposed to start proper research. From then on, these yearly meetings are supposed to provide an opportunity for the committee to help keep the student on track - kind of like annual checkups. I'll let you judge for yourself how much value could possibly be provided to an active research student in terms of education by even the most amazing committee in the span of 90 minute discussions that take place once per 365 days.
So, when it comes to committees at least, I disagree with your assertion that the problem lies with their personal failings to meet their responsibility to educate the next generation of scientists. The problem isn't them, its the system we have here - a system that does nothing to reward them for their time, suggests that 90 minutes per year per student is enough to help the student advance, and puts incredible pressure on them to achieve in other aspects of their careers. In my situation, I only blame them for making a promise of timeliness they should have known that they would not be able to keep.
We shouldn't complain about the failure of faculty. We should figure out ways to create better systems for education - and I think such systems will orient around finding better ways to distribute expert attention ;).
As regards to the closing comment that we should not complain about failure of the faculty, may I ask you who made the program that decides that one meeting of about 90 minutes is enough for a year? Who are the ones that committed time they are not willing to set aside for it? Who decided they do not get any incentives for doing thesis committee work?
Superficially, blaming “the system” is a safe and easy third party, so that everybody can absolve himself from responsibility because there are always excuses to find in the murky ongoings of “the system”, but people should realise we make the system, are the system, and perpetuate the system. The programme did not come out of the blue. Your university is around longer than the one I work for. The year I started was the first year of a PhD programme of which the structure and a new system had to be put in place. It was not possible to copy from another department, but it all had to be invented and negotiated from scratch amongst the professors here. There was no system; it had to be formed, shaped, and being reshaped. That power the faculty does have. Students can complain and come up with ideas to improve it--as we did, is being continued and expanded upon now--but it is the faculty that decides on the program and the system that implements those regulations. If one observes that “the system sucks”, it is the faculty that has the position, expertise, and managerial power to change it. (Then there is the larger system of academistan, but deserves its own post.)
So yes, I disagree with you trying to find excuses for them not doing their work.
And, in my humble opinion, I think it would have been more appropriate to channel your complaints within the uni compared to posting it on your blog, but then, maybe “the system” at your uni does not have such feedback channels. yet.
Multiple things have been experimented with across different universities that tend to work. For instance, official recognition for being member of the thesis committee (compared to something like ‘just to meet administrative obligations as the programme regulations say so’); sharing the workload (e.g., each member reads intro, conclusions, and one chapter); taking into account such activities in promotions and tenure (cf., say, just publications); longer-term track record of where the graduates end up; student feedback, be it during the programme or an exit questionnaire.
While they could contribute to shifting patterns of expert attention, one, perhaps, could change the rule of the game by ditching the paradigm of the demand-economy where expert attention is a scarce good one must beg and pay for.
Hey Ben,
One thing that I constantly remind myself of when making cold calls - remember how it feels to receive a cold call.
We are all short on time these days but people still want to help each other out for good karma / what goes around comes around, etc.
Persistence is good and also not taking it personally when someone doesn't respond the way you are hoping they would have. Keep at it!
Professor Keet,
I suppose you are right - if my intention was to make local improvements to the educational system I am a part of here, then I could probably generate more immediate response by going directly to the program founders and arguing my complaints. Just as I might improve my local environment by selling my car and attempting to live completely on solar power. Think global act local right? Umm.. no not really. I'd rather invent a new way to generate power - too much science fiction in my upbringing I guess.
For me, I'm much more interested in understanding and creating possibilities for more fundamental change than I am in fighting the local behemoth bureaucracy from the extremely weak position I now occupy. In your comment, I want to understand how the "larger system of academistan" might be changed for the better. I look forward to your post about that!
I would also like to hear you clarify what you mean by this: "ditching the paradigm of the demand-economy where expert attention is scarce". How might that work???
Hi Matt ! Nice to see you here! I'll keep pushing while trying not to be to pushy ;). Good luck with TourVista!
There is nothing wrong with thinking big, but for coming up with an implementable "grand plan" as you seem to be looking for, it helps getting experience how things work and can [or cannot] be changed at a smaller scale—be it organising PhD students at the university (e.g. here), nationally (adi), and internationally (eurodoc), and within the structures as member of the various gremia.
The posts on the other two topics will not be online next week already—it’s conference submission deadline season right now....
Update. In the end, everyone came through in time for me to submit the dissertation this spring. Yay! (Pestering pays...)
Now what on Earth am I going to do with myself???
Post a Comment