The "late surprise" problem

[Brian E Carpenter]

Actually this is a solution proposal, which Dave Crocker and I have
been discussing the last couple of days. I think it has value, and
it also illustrates what I think we have to do - find subsets of
the total problem for which we can rapidly implement incremental
solutions.

   Brian-------------- next part --------------
Avoiding Late Surprises
=======================

This is a strawperson proposal for a way to solve one particular
problem in the functioning of the IETF.

This was put together by Brian Carpenter and Dave Crocker, but
draws on ideas in the air such as Ted Hardie's "stuckee" draft
and efforts in some WGs (e.g. IPv6) to formalize document review.
It is also somewhat inspired by the MIB Doctor system.

1. Problem Area

Working Group specifications do not receive formal review until 
they are submitted to the IESG. Hence, significant problems with 
a specification often are not detected until considerable effort 
has been wasted and changes to fix the problems are difficult to add.
We can speculate that this problem explains a large part of the
long tail in the distribution of IESG document approval times.

The same applies equally or more so to individual submissions.

2. Formalized pre-IESG review

Create a panel of recognized Senior IETF Reviewers (SIRs). Note that 
this is an IETF-wide pool, not Area-specific. See below for membership.

Require that at least N of these people provide periodic technical
reviews of the progress of each working group.

Require that at least N of these people perform a formal 
technical assessment of each working group document prior to its
submission for IESG approval. In other words, WGs are simply not
allowed to submit a draft to the IESG unless accompanied by N reviews.

Require that at least N of these people perform a formal 
technical assessment of each independent submission prior to its
submission to the RFC Editor. In other words, individuals are simply not
allowed to submit a draft unless accompanied by N reviews.

We should discuss whether the reviews should include a recommendation.

For WG drafts, it's clearly the WG Chair's job to ensure that the reviews
are done. For independent submissions, the author should personally have the
reviews done before submitting to the RFC Editor.

How large is N? 

For WG progress reviews- let's say 5.
For standards track documents or BCPs- let's say 5.
For experimental or informational - let's say 3.

3. Who gets to be a Senior IETF Reviewer?

This should be a formal IETF position that people can
put on their resume if they want to. Let's have a goal of 50 to
100 people. The pool of SIRs will be listed on the IETF web site,
preferably with some Web tooling to manage the review process
(similar to or integrated with the ID Tracker).

To bootstrap the membership, appoint an initial set consisting of 
current IAB members, all available and willing former IAB and IESG 
members and WG Chairs, and existing MIB doctors.  Maybe include available 
authors of some minimum number of RFCs. Maybe include Directorate members 
where there are directorates - remember however that this is to be
a cross-Area review panel.

Ongoing membership: have an annual process for the current set of SIRs to
elect a few more SIRs, until the number reaches 100.

SIRs who do not conduct an agreed minimum number of reviews in a year
will step down.

In extremis, SIR status may be removed by a simple majority vote of
the panel of SIRs.

--