what is a problem

[John C Klensin]

--On Monday, 25 November, 2002 13:17 -0500 Thomas Narten
<narten@us.ibm.com> wrote:

> presnick@qualcomm.com (Pete Resnick) writes:
> 
>> A while back, I had a document "stuck" in IESG review for
>> what I  thought was a long period of time. It was
>> (apparently) due to one AD  who had a concern with the
>> document and I couldn't find out the  specifics of the
>> problem.
> 
> Note: this is not supposed to happen (but I'm not saying it
> never has). When a document is under full IESG review, issues
> with documents need to written up and communicated back to the
> authors/WG. Our internal policies have required this for some
> time, as formalized by IESG "Evaluations".

Thomas, I hope that Pete's situation and experience is very
different from mine.  But let me throw another example into the
pot.  

Some years ago, the IESG gave itself the authority to review
"independent submissions" to the RFC Editor to be sure that they
did not constitute an end-run to WG activities.  At a later
date, that authority was expanded to reviewing documents to be
sure that they did not contain protocol suggestions that could
be significantly damaging to the infrastructure.  In both cases,
the intent was to generate a recommendation to the RFC Editor,
either to not publish or to publish with an "IESG Disclaimer".
Neither decision was published for review by the community: that
is probably not a major problem (I, at least, consider those
reviews to be reasonable and in everyone's best interest), but
it may be symptomatic.  In both cases, the IESG was to have been
given a fixed period of time to do the review, with a default of
"publish" (more or less consistent with Geoff's and Marshall's
recommendation vis-a-vis standards and, to me at least, much
more obviously reasonable for these cases).

As a relevant aside, around 18 months or two years ago, in an
attempt to reduce IESG workload on non-standards-track, non-IETF
materials, the RFC Editor was asked to perform a preliminary
review on individual contributions before those went to the
IESG.  The intent was to avoid having the IESG spend time on a
document that the RFC Editor would decline to publish anyway.
But, at least according to the RFC Editor, the IESG then
instructed them to give priority to anything, including
individually-submitted informational document coming to them
from the IESG.  This created, in practice, a two-track
situation, in which individual documents could be submitted
either to the IESG or to the RFC Editor.  The latter went into a
black hole, making it obvious that anything anyone wanted
published should go to the IESG.  Several requests, circa last
July, that the IESG document the types of individual documents
it wanted submitted directly, which documents should go to the
RFC Editor first, and what guidance was given to the RFC Editor
about priorities for handling, were essentially ignored.

This is one of the many examples that had led some of us to call
for an "IESG Charter" or other document that would identify what
the IESG considers the appropriate scope of, and limits to, its
responsibility and authority.

But, continuing with the example, the IESG has now, apparently,
expanded its oversight of individual submissions to include
negotiation with authors about what those documents say.  Hence
we have documents that make no normative statements, do not
specify anything that could be construed as a protocol, and that
have been exposed to several rounds of comments through the I-D
process, go into the state "Point raised - writeup needed".  And
there they sit, sometimes for an extended period of time.  No
clue to the author as to what the issue is.  And remember that,
unless the IESG has quietly expanded its role (again), the only
possible comments on this type of document are supposed to be
"ok to publish if the RFC Editor wants to do that", "end-run
around the foo WG's efforts/charter or around standards-track
RFC ZZZZ", or "damaging if deployed".   If either of the latter
were invoked, the status would presumably indicate that the IESG
was waiting for disclaimer text and the author (and, probably,
RFC Editor) would already have been informally notified.
Instead, we have a procedure that apparently permits IESG
members to drop things into black holes.  And, while the tracker
permits us to identify the "shepherding AD" (which is real
progress), it doesn't permit us to identify the AD who is
sitting on the thing.

>> Now, let's imagine the current IESG process 
>> is "If an AD wants to hold up publication of a document, they
>> have to  provide the shepherding AD with a detailed
>> explanation of the problem  to convey back to the document
>> editor." If that were the case, then  the problem might be
>> particular ADs not following the procedures of  the IESG.
> 
> The above is what is supposed to happen (and does mostly). But
> there have certainly been times in the past when the ball got
> dropped during this step. One can blame the AD who has the
> issue for not writing it up, and/or the shepherding AD for not
> doing enough nagging to get the comments out, or the
> shepherding AD for not taking the comments and actually
> delivering them back to the authors. All have happened at
> times.

And I note that none of the above requires that an AD who raises
an objection and promises to write it up do so within any fixed
time.  One of the many places where my understanding of these
situations converges with that of Geoff and Marshall is that, if
someone has an objection, they should get a fixed (and
relatively short, with length perhaps a function of the document
type) to write it up.  If they don't do so within that period,
they should rationally be treated as not caring enough, or not
having clear enough ideas, and disregarded.

This is especially true when the content of an
individually-submitted informational document is involved, since
the RFC Editor is understood to be willing to accept, nay
encourage, "foo is a bad idea" or "bar is harmful" documents
that respond to already-published position statements.

> Note that ID tracker will certainly help avoid this sort of
> problem. Personally, it's now very easy for me to see which
> documents (of the ones I'm shepherding) are in the "IESG
> Evaluation" state, and/or those that are waiting for discuss
> comments to be written up and so on.

Yes, but, if my understanding is correct, you also get to see
"who is sitting on it, and how long have they been doing that".
We don't.
 
>> We can all think of solutions to that problem. However, let's
>> instead imagine that the current process is "If an AD wants
>> to hold up publication of a document, they may ask the IESG
>> to 'discuss' the document, and that can go on ad infinitum."
>> If that were the case, then the problem might be that the
>> IESG process allows documents to languish on the say-so of
>> only one AD. That's a different problem with a much different
>> set of solutions.
> 
> IESG internal policies (i.e., running code) do not allow an AD
> to have a vague "I don't like this" type of discuss. There is
> a little give and take here, of course. The AD with the issue
> needs to describe an issue sufficiently so that it can be
> taken back to a WG/author. But the Shepherding may also want
> to push back and say "I don't understand this", or "this is
> too vague", etc. But this back-and-forth is not open ended.

Thomas, the evidence that it is not open ended is, to put it
mildly, inconclusive.
 
> It is also the case that when a document goes to the full
> IESG, we discuss it at the next telechat. During the telechat,
> all issues are supposed to get flushed out, modulo the "defer"
> (for more time). But even a defer is a one-time event.
>...

    john

p.s. Some of the comments above are likely to reopen the
question of why we need to have individual-submission
informational documents published through the RFC process at all
in this age of web publication and multiple archives.  I don't
want to spend significant time on it unless others feel that
topic is important, but my own view is that such documents can
serve a useful role in documenting paths not taken and in
providing perspectives on the applicability of protocols.  I
wish IETF were doing a better job of creating clear and
well-documented applicability statements, but we aren't, and, in
their absence, well-reasoned and appropriately-identified
individual perspectives seem to me to be very useful... and the
RFC series seems to me to be the right place to publish them.
Personally, I could live without RFCs that have nothing to do
with IETF/IRTF activities, but I would hope that, if we need to
make that distinction, we can construe it generously.  But, as
long as we publish, as RFCs, informational or experimental
documents that are anything other than WG-approved, consensus,
supplemental materials to standards-track documents, I think
there is likely to continue to be an important role for
individually-produced documents that comment on them or provide
alternate points of view.