General comment on draft-ietf-problem-statement-00.txt

[Keith Moore]

On Fri, 28 Feb 2003 12:13:11 -0600
"Spencer Dawkins" <sdawkins at cynetanetworks.com> wrote:

> Hi, Keith,
> 
> I'm asking, not arguing - are you suggesting we make a pass through the 
> problem statement to see if we've specified a solvable problem, and 
> then nail the solutions team to solving the problem statement problem?

no.  I'm only suggesting that getting a useful problem statement is likely to
require multiple iterations - requiring input both from within the WG and from
outside.

> My thinking was that the solutions team would (likely) be solving a 
> subset of the problems we've identified (modulo boiling the oceans and
> creating world peace without hunger). I'm guessing Keith thought I was
> opening the door to solutions for NEW problems that weren't identified
> in the current draft (or its successors). I wasn't, for the reason
> Keith mentioned.

not my thinking at all, sorry if I wasn't clear.  by 'things that are
not problems' I was referring to (for example) the idea that one or two IESG
members can 'veto' a document.   I realize that several people have that
impression, but it's a false one or at best a misunderstanding. furthermore,
trying to 'solve' this 'problem' is likely to make things worse - by lowering
the quality of IETF's output and/or by increasing IESG workload.  this kind of
statement is worse than either a statement about what is observed ("documents
are often delayed for long periods of time over objections that seem to be
coming from only one or two IESG members")  or a detailed analysis of what is
happening to produce such delays.

For instance:

- due to IESG workload and breadth of IETF's scope, most documents are only
reviewed in detail by one or two IESG members anyway, so objections from one
or two members necessiarly carry more weight than it would otherwise seem. 
It's not unusual for an AD to say "I trust the ADs in that area to do due
diligence in fixing that document".  This tends to mean that *any* objections
to a document, no matter how valid, come from only one or two ADs.   (if you
get more than two or three ADs with substantive objections to a document this
is a sign that the document has attracted far more than the usual degree of
attention and that it is widely viewed as a problem within IESG.  even then,
veto of documents that come from a WG is extremely rare.)

- Sometimes an AD objects to something in a document because he doesn't
understand some subtle aspect of the design or the compromises that were
deemed necessary by the WG.  Given workload and scheduling constraints (and
sometimes stubbornness) it can (quite unfortunately) take months to work these
things out.

- Sometimes one AD attempts to slip something past other ADs but only
one or two ADs notice and are willing to invest the time/energy to object.

- the nature of the IESG decision-making process is that ADs who object to a
document are expected to explain what it will take to fix the document. 
sometimes (too often!) the protocol is so flawed, or the document so poorly
written, that it simply can't be fixed by making a few tweaks.  so the ADs who
want to be conscientious (and not simply approve the document) end up making
vague or seemingly trivial objections, because there isn't time enough for the
AD to rewrite the document.  (and the WG doesn't know what to do with these
objections).  other ADs just hold their noses and vote 'no objection' because
providing useful review and feedback given the constraints of the balloting
process is either infeasible or too time-consuming. 

- when a WG submits a document to an AD, that AD has to vote 'yes' on the
document in order to get the document on IESG's ballot.  what happens if that
AD has problems with the document?  he iterates with the WG.  what happens if
the AD feels that the document is so bad that it can't easily be fixed?  the
process stalls.  the AD needs more time to come up with a set of fixes that
the WG will agree to before the AD can endorse it, and the AD's time is in
short supply.  the longer the document(s) in question, the more time it takes.

there are even more aspects to this 'problem', but perhaps these are enough
to illustrate the necessity of digging deeper into the problem definition
phase of this effort.

Keith