The "late surprise" problem

[Brian E Carpenter]

Ted, I think that if a review panel existed, WGs should be 
encouraged to use it early in the process as well as late. But
it's hard to make that mandatory.

In answer to Pekka - the review is consciously not a yes/no
decision. So mixed reviews would be a signal for detailed IESG
attention. All good or all bad reviews would give the IESG a strong
hint for a quick decision.

I won't respond to the other comments sent so far, but they are all good
input.

   Brian

Ted Hardie wrote:
> 
> Brian,
> 
> This may solve a problem, but I don't think it quite addresses
> the same problem I've tried to address.  It creates a review
> body that would no doubt be of enormous help to the IESG,
> which might speed things along.  It is, however, still very
> late in the process.  Fundamentally, it is after the working
> group process finishes.
> 
> I would really like us to look at the problem in the working groups.
> I'm afraid the solution proposed might continue the existing problem
> that
> working groups do not feel empowered to make decisions.  I also think
> that it might continue the issue that it will be difficult to identify
> who
> in a working group has committed to taking on a particular
> set of work.  Though the stuckees draft talks specifically about
> the review task, the problem is general.
> 
> Fundamentally, I believe that if we improve the way working
> groups get their tasks done, the review task will look less
> like a blocking factor.  That doesn't mean a "program committee"
> style pool wouldn't be a valuable thing--it very well might.  But
> I don't think it solves the same problem, and I would be very
> concerned if it replaced work on making the working groups
> more effective and empowered.
>                                         best regards,
>                                                         Ted Hardie
> 
> 
> On Thursday, March 20, 2003, at 09:39 PM, Brian E Carpenter wrote:
> 
> > Actually this is a solution proposal, which Dave Crocker and I have
> > been discussing the last couple of days. I think it has value, and
> > it also illustrates what I think we have to do - find subsets of
> > the total problem for which we can rapidly implement incremental
> > solutions.
> >
> >    BrianAvoiding Late Surprises
> > =======================
> >
> > This is a strawperson proposal for a way to solve one particular
> > problem in the functioning of the IETF.
> >
> > This was put together by Brian Carpenter and Dave Crocker, but
> > draws on ideas in the air such as Ted Hardie's "stuckee" draft
> > and efforts in some WGs (e.g. IPv6) to formalize document review.
> > It is also somewhat inspired by the MIB Doctor system.
> >
> > 1. Problem Area
> >
> > Working Group specifications do not receive formal review until
> > they are submitted to the IESG. Hence, significant problems with
> > a specification often are not detected until considerable effort
> > has been wasted and changes to fix the problems are difficult to add.
> > We can speculate that this problem explains a large part of the
> > long tail in the distribution of IESG document approval times.
> >
> > The same applies equally or more so to individual submissions.
> >
> > 2. Formalized pre-IESG review
> >
> > Create a panel of recognized Senior IETF Reviewers (SIRs). Note that
> > this is an IETF-wide pool, not Area-specific. See below for membership.
> >
> > Require that at least N of these people provide periodic technical
> > reviews of the progress of each working group.
> >
> > Require that at least N of these people perform a formal
> > technical assessment of each working group document prior to its
> > submission for IESG approval. In other words, WGs are simply not
> > allowed to submit a draft to the IESG unless accompanied by N reviews.
> >
> > Require that at least N of these people perform a formal
> > technical assessment of each independent submission prior to its
> > submission to the RFC Editor. In other words, individuals are simply
> > not
> > allowed to submit a draft unless accompanied by N reviews.
> >
> > We should discuss whether the reviews should include a recommendation.
> >
> > For WG drafts, it's clearly the WG Chair's job to ensure that the
> > reviews
> > are done. For independent submissions, the author should personally
> > have the
> > reviews done before submitting to the RFC Editor.
> >
> > How large is N?
> >
> > For WG progress reviews- let's say 5.
> > For standards track documents or BCPs- let's say 5.
> > For experimental or informational - let's say 3.
> >
> > 3. Who gets to be a Senior IETF Reviewer?
> >
> > This should be a formal IETF position that people can
> > put on their resume if they want to. Let's have a goal of 50 to
> > 100 people. The pool of SIRs will be listed on the IETF web site,
> > preferably with some Web tooling to manage the review process
> > (similar to or integrated with the ID Tracker).
> >
> > To bootstrap the membership, appoint an initial set consisting of
> > current IAB members, all available and willing former IAB and IESG
> > members and WG Chairs, and existing MIB doctors.  Maybe include
> > available
> > authors of some minimum number of RFCs. Maybe include Directorate
> > members
> > where there are directorates - remember however that this is to be
> > a cross-Area review panel.
> >
> > Ongoing membership: have an annual process for the current set of SIRs
> > to
> > elect a few more SIRs, until the number reaches 100.
> >
> > SIRs who do not conduct an agreed minimum number of reviews in a year
> > will step down.
> >
> > In extremis, SIR status may be removed by a simple majority vote of
> > the panel of SIRs.
> >
> > --
> >