The "late surprise" problem

[John C Klensin]

--On Friday, 21 March, 2003 20:09 +0100 Jeanette Hofmann
<jeanette at wz-berlin.de> wrote:

> In the academic world, peer review itself forms a
> bottleneck. First of all,  reviewing other people's work
> takes a lot of one's productive time. Reviews  always
> compete against more urgent duties. People tend to postpone
> them. I  don't know anybody who really enjoys review jobs. 
> 
> Second, reviews turn easily into a weapon against a certain
> approach, school  of thought or person. Not even blind
> reviews are able to prevent this. 
> 
> A panel of reviewers would create a new hierarchy level in
> order to filter out  all sorts of noise. Potential costs
> might be a further slow down of the process  and wrong
> filtering mechanisms. It might also be a problem to find
> enough  competent people who are actually willing to perform
> this job. 

Jeanette,

I'd add one thing to this list, which is another reason I
believe we need to really think through the side-effects of an
institutionalized cabal.    Peer review processes in
relatively small communities tend to get very incestuous.
People who are part of the review circle tend to be nice to
each other's documents and proposals, understanding that their
own products will be reviewed tomorrow by those whom they are
reviewing today.  And "outsiders" often have a really hard
time getting in, especially if their inclusion would reduce
the resources being shared by the in-group pool.

Again, I'm not against the proposal, I just think we need to
understand what other problems it might cause (or worsen) and
decide whether those are worth it.

      john