Document Blocking (Was: I-D

[John C Klensin]

A few observations on "document blocking".  Others are covering 
the "discuss" theme, but there is another one that might be 
slipping toward the cracks...

In his note of Friday, 16 May, 2003 13:03 -0400 Scott Bradner 
<sob at harvard.edu> wrote:

> some background on the blocking documents thread based on my
> experience on the IESG
>
> lets split documents that the IESG looks at into two piles
> 	1/ documents forwarded to the IESG by the RFC Editor to get
> the IESG's recommendation
> 	2/ documents that are the product of IETF working groups or
> independent documents (mostly for standards track) that are
> being evaluated as if they were WG documents
>
> case 1:
> the document is normally assigned to a specific AD - that AD
> does an evaluation and comes up with a suggestion as to what
> the IESG should say to the RFC Editor - this can take an
> arbitrarily long time
>...
> sometimes the AD will work with the document authors/editors
> to deal with issues that the AD finds during their review,
> sometimes this happens before the document getting on the IESG
> agenda sometime after the IESG discussion

And sometimes the AD has other things on his or her mind, and 
this can take an arbitrarily long time no matter how responsive 
the author is.

> it is not common for the IESG to recommend that the RFC Editor
> not publish a document (but it does happen a few times a year)
> - when it does happen the RFC Editor is sent a 'do not
> publish' (DNP) recommendation.  The DNP is written by one or
> two ADs - this can take an arbitrarily long time
>...

> case 2:
> it is quite common for issues/questions to be raised by one or
> more ADs during the IESG evaluation of a document - if the
> AD(s) feel strongly enough that there is an issue that needs
>...
> to be addresses they vote "discuss" - this will block a
> document until the AD(s) are satisfied by revisions in the
> document or as a result of discussion
>
> when an AD has an issue with a document and has voted
> "discuss" the document and the issues are discussed during an
> IESG teleconference - sometimes the discussion results in the
> AD changing their evaluation and removing their "discuss"

And sometimes, it results in the AD saying, more or less, "I 
will explain my reasons in a note or draft writeup".  At that 
stage, unless there is a clear consensus that the AD is off 
base, the IESG will give the AD as much time as needed to do 
that writeup (after all, each AD might be in the same position 
next time).  That can take an arbitrarily long time.

> I do not recall that the IESG has ever specifically decided to
> not publish a document in this group but since it can take an
> arbitrarily long time for the working group (or document
> authors/editors) to respond to the concerns and an arbitrarily
> long time for the back and forth between the AD for the
>...

Also, Keith raised a question and Scott responded on Friday, 16 
May, 2003 15:27 -0400:

>> does the RFC editor no longer impose a timeout on IESG
>> feedback for such documents?  (where such feedback could say
>> "we need more time" or "we're discussing with the author")
>
> formally, they do, but it is meangingless

And Keith later wrote, on Friday, 16 May, 2003 16:27 -0400:

> Right, but IESG has timeouts in place for almost all phases of
> its deliberative process.   It's the WG that can hold up
> things for arbitrary amounts of time (sometimes because the
> authors and/or chair are sitting on the document and the WG
> doesn't know what is happening, sometimes because the WG
> refuses to make changes and doesn't know what to do next).

Now, my first observation is that phrases equivalent to 
"arbitrarily long time" appear much too often above, especially 
when they are linked to experience in which things have taken a 
very long time.   There is certainly a difference procedurally 
and formally between blocking by "discuss"  and "just tie it up 
for an "arbitrarily long time" while something is ostensibly 
being done.  But, in practical terms, there really is no 
difference at the end of the day: the document isn't getting out 
and isn't getting cleanly rejected with clear reasons either.

Part of the problem is that, as with the "RFC Editor timeout" on 
non-WG/ non-standards track submissions (Scott's "case 1"), the 
timeouts to which Keith refers are, in my observation, routinely 
ignored.

Certainly sometimes, probably often, the delays can be pinned on 
the WGs as Keith suggests.  But there is a loop in that 
particular bit of finger-pointing too.  Most of us who have 
observed or worked with long-lived WGs have noticed that it is 
possible for them to reach a point of exhaustion in which 
everything happens slowly, if at all, getting a reasonable 
number of people to focus on an issue or decision or to 
carefully review a document.  So, suppose I were an AD holding 
the token on a WG document and wanted to _cause_ that to happen 
(for the record, I'm not aware of any cases in which this has 
been done deliberately.  But the intent is less important than 
the effect).  I would send the WG a long-ish list of things that 
I insisted be fixed, some important, some trivial, and at least 
a few guaranteed to irritate them (possibly by raising old 
issues that the WG thought it had already become exhausted 
discussing).  I would wait for a response, possibly a whole new 
draft.   I might insist that the new draft be Last Called within 
the WG, just to be sure that consensus still existed.  If the WG 
asked me questions about my position, I'd take my time 
answering.   I'd count all of the time the WG spent on that 
process as "WG delay" and the current structure of the tracking 
system would help me document that.   Then I'd do it again -- 
I'd review the revised document, find more stuff to object too 
(even if it was completely trivial) and send the document back 
again.  If the WG Chair and Editor(s) shared the issues with the 
WG, the odds are good that frustration and exhaustion would set 
in quickly.   If they didn't, but tried to negotiate the trivia 
(remembering that opinions about what is, and is not, trivial 
can vary widely depending on one's perspective) privately, but 
that took a while (it always does), the WG members would get 
frustrated about the apparent "black hole" situation.

My conclusion is that we need to have timeouts, and they need to 
be meaningful.   Meaningless timeouts may be worse than no 
timeouts at all.  The problem is how to make, and keep, them 
meaningful, especially when the rules we already have are --it 
seems regularly-- ignored.  And we should figure out ways to do 
that which don't compromise quality of review.  Examples (I'm 
not really proposing solutions here, just possible ways to think 
about the problem):

* After an AD gets a document from a WG with a request for Last 
Call, or after Last Call completes, he or she has N weeks to get 
the Last Call issued or to get the document in front of the 
IESG.  The document may, instead, be returned to the WG during 
that window with an explanation.   But, if it is neither 
returned nor processed within N weeks, the AD must provide a 
public explanation of why the added delay is needed.  And, at 2 
or 3 times N, there needs to be IESG consensus --and another 
public statement-- as to the importance and reasonableness of 
more delay.

* For "case 1" documents, remember that "sometimes the AD will 
work with the document authors/editors to deal with issues that 
the AD finds during their review" (from Scott's explanation 
quoted above)is a process violation.  For those documents, the 
IESG is permitted only to tell the RFC Editor that there is an 
end-run going on, or that the suggestion, if implemented, would 
be harmful to the Internet.   "Make an AD happy" or "resolve any 
'issues' the AD has" are not on that very short list.  Certainly 
we should not discourage an AD who has the time and interest 
from trying to sort "issues" out with an author.  But, until and 
unless we change the procedures, that should occur by the AD 
--more or less as an individual member of the community-- saying 
to the author "I've got some problems with this and would like 
to work with you on them".  The author can then pull the 
document out of the queue and get to work.   Or he or she can 
decline to do that, at which point the AD --again, acting 
strictly as an individual-- could reasonably tell the RFC Editor 
that the discussion/interaction occurred as input into the RFC 
Editor decision about publication.  But an AD's sitting on an 
individual submission to "work on issues" under the umbrella of 
doing "an evaluation and com[ing] up with a suggestion as to 
what the IESG should say..." is abuse of authority.  There will 
certainly be times when the initial "timeout" period is 
insufficient to conduct even the minimal review that 2026 
requires and permits.   But, then, let's get a public notation 
that a delay has been requested and an explanation for it: it 
might help move things along, or, at worst, provide the 
community with a paper track to help evaluate whether the 
particular AD should continue to hold that position.

     john