The "late surprise" problem

[Dave Crocker]

SWB> I don't see how this is better than using enhanced directorates.

1. scope of the pool is different:  area-constrained vs. ietf-wide

2. selection of the pool is different: chosen by working group vs. solely
chosen by and accountable to the area director

3. relationship with the working group is different: none vs.
essentially part of the wg.

There are probably more, but 3 major ones seems like a good start.
They demonstrate fundamental differences in relationship to the
process.


SWB> The advantage of directorates is that they are managed by the
SWB> ADs. (That looks like an advantage to me -- it might not look
SWB> like one to you if you are trying to strip power from them.

Area directorates are not supposed to have "power". They are staff
advisors to area directors, solely chosen by, and accountable to, area
directors. The SIR proposal is for entirely different selection,
accountability and authority.



JH> In the academic world, peer review itself forms a bottleneck.

This proposal has marked differences from typical peer-review
processes.  The selection and interaction bases are entirely
different, notably because peer review is typically managed by the
higher authority body.  Further, their work occurs only during
submission time.  Hence, peer review correlates well with current IETF
Area Directorate work.

By contrast, the SIR proposal permits working groups to manage their
own process.  The sign-off at the end is the "graduation prize" but
would be most intelligently achieved by involving the desired SIR
signatories early and often in the working group's process.

(I had wanted to call them senior "contributors" but Brian came up
with the SIR acronym and it is vastly superior -- not to mention that
the alternative,for mine would be, at best, distasteful.)


JH>  First of all,
JH> reviewing other people's work takes a lot of one's productive time.

And here is the part that appeals to me most: An intelligent working
group will not simply come to these reviewers, hat in hat, pleading
for a review at the end of the specification effort.

They will, instead, involve these SIRs in the development process.
That will get better contributions and MORE readings.


JH> Second, reviews turn easily into a weapon against a certain approach, school
JH> of thought or person. Not even blind reviews are able to prevent this.

That is why it is essential that the working group have control over
selection of the particular reviewers.


JH> A panel of reviewers would create a new hierarchy level in order to filter out
JH> all sorts of noise. Potential costs might be a further slow down of the process
JH> and wrong filtering mechanisms. It might also be a problem to find enough
JH> competent people who are actually willing to perform this job. 

And in spite of my enthusiasm, above, I'll acknowledge that each of these
points you make does have some potential.



HTA> - colorize the SIRs. Routing clue has a green dot, internationalization
HTA> clue has a blue dot, security clue has a black dot and so on - 5 or 6
HTA> categories. Require that reviewers have at least 3 different clues between
HTA> them.
HTA> One important property of review HAS to be that documents get reviews from
HTA> someone who is NOT married to that particular religion.
HTA> Matching problem statement:

These are appealing refinements.  My primary question is about the
effort it will take to define the details, achieve community consensus
on them, and administer the on-going operation.


HTA> - Make sloppy reviews have an effect on SIR status - if some reviewer
HTA> (whether SIR or not) detects a problem that a SIR *should* have caught,
HTA> transfer the dot to the guy who caught it.

This is beginning to sound like a rather sophisticated, new "dot"
economy.

The proposal has a way to removing SIR status. Do we really need
something different from that simple mechanism?

d/
--
 Dave Crocker <mailto:dcrocker at brandenburg.com>
 Brandenburg InternetWorking <http://www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>, <fax:+1.866.358.5301>