General comment on draft-ietf-problem-statement-00.txt

[Pete Resnick]

On 2/28/03 at 2:03 PM -0500, Keith Moore wrote:

>by 'things that are
>not problems' I was referring to (for example) the idea that one or two IESG
>members can 'veto' a document.   I realize that several people have that
>impression, but it's a false one or at best a misunderstanding. furthermore,
>trying to 'solve' this 'problem' is likely to make things worse - by lowering
>the quality of IETF's output and/or by increasing IESG workload.  this kind of
>statement is worse than either a statement about what is observed ("documents
>are often delayed for long periods of time over objections that seem to be
>coming from only one or two IESG members")  or a detailed analysis of what is
>happening to produce such delays.

Though I disagree with you and Brian about certain things not being 
problems, I do find it interesting that your analysis below 
identifies real and serious problems, some of which might not be 
addressed in the current draft, but most of them are:

>- due to IESG workload and breadth of IETF's scope, most documents are only
>reviewed in detail by one or two IESG members anyway, so objections from one
>or two members necessiarly carry more weight than it would otherwise seem.

Well, this is a straightforward statement of the workload problem. 
You're saying that the IESG is so overworked that finding a problem 
in a document is rather lucky: Only a couple of IESG members will be 
reviewing it anyway, so if they happen to be the ones that have the 
expertise to find problems, we ought be glad they caught it.

Workload then seems like something that needs to be addressed. It is 
mentioned specifically in A.4, but I think 2.2 is supposed to be the 
root of it (i.e., the IESG would have less workload if the IETF used 
different practices).

>It's not unusual for an AD to say "I trust the ADs in that area to do due
>diligence in fixing that document".

Why is it then that AD's are not inclined to say "I trust the chair 
of that working group to do due diligence in fix that document"? 
Perhaps this indicates another problem, that we do not have 
adequately qualified WG chairs that ADs can trust to do the real 
review (cf. 2.6)? This does seem like an area where "too much 
concentrated authority" (lack of delegation) is part of the problem. 
(2.4)

>- Sometimes an AD objects to something in a document because he doesn't
>understand some subtle aspect of the design or the compromises that were
>deemed necessary by the WG.

This may be the fault of a document editor not appropriately 
documenting a compromise choice so that AD can figure out why the 
choice was necessary. That may amount to a 2.5 kind of problem.

>Given workload and scheduling constraints (and
>sometimes stubbornness) it can (quite unfortunately) take months to work these
>things out.

Though I understand it was parenthetical, I am quite concerned that 
"stubbornness" was a feature of the process that you thought worth 
mentioning. I understand that ADs are human beings and have the same 
flaws as the rest of us. However it seems absolutely essential that 
we choose ADs who can get by certain human failings and say, "The AD 
who put this forward is not stupid, yet I still think there is 
something flagrantly wrong here. Perhaps I should go talk to the 
chair or the document editor to see what's going on before I assume 
that I'm the only sane one on the IESG." I have had personal 
experience where this did not happen and it held up a document for 
months. I am worried that we are not recalling ADs who are too 
"stubborn". They are an impediment to good quality work being done in 
a timely fashion.

>- Sometimes one AD attempts to slip something past other ADs but only
>one or two ADs notice and are willing to invest the time/energy to object.

OK, so my chin hit the keyboard when I read this. Surely you're 
kidding. If we've got ADs trying to slip something past other ADs, 
those folks should be instantly recalled. You're not talking about 
negligence here (where an AD might not be diligent enough with their 
review), but someone who knows that something is wrong or might get 
objections from other intelligent and responsible ADs and tries to 
slip it by anyway. Now, maybe you're saying they have to do this 
because of incorrect behavior on the part of the other ADs, like 
objecting to documents without justification, but in that case those 
bad behaving ADs ought to be removed. Short of that, an AD who is so 
irresponsible that they try to slip something by on the others should 
be out on their rear immediately.

In addition, it seems like if one or two ADs *do* notice such an 
occurrence, it should be  easy for those one or two to convince the 
rest of the ADs what's going on. In that case, we shouldn't be seeing 
1 or 2 discuss votes, we should be seeing 12, shouldn't we?

Did I miss something in what you meant?

>- the nature of the IESG decision-making process is that ADs who object to a
>document are expected to explain what it will take to fix the document.
>sometimes (too often!) the protocol is so flawed, or the document so poorly
>written, that it simply can't be fixed by making a few tweaks.

Those seem like 2.2 problems.

>so the ADs who
>want to be conscientious (and not simply approve the document) end up making
>vague or seemingly trivial objections, because there isn't time enough for the
>AD to rewrite the document.  (and the WG doesn't know what to do with these
>objections).  other ADs just hold their noses and vote 'no objection' because
>providing useful review and feedback given the constraints of the balloting
>process is either infeasible or too time-consuming.

So to address (compound?) the 2.2 failures, most ADs hold their noses 
and let the document go through (that's pretty irresponsible) and one 
or two ADs vote discuss, but don't make reasonable objections to the 
document. That's horrible.

However, it seems to me that the document gets right at the core 
problem in 2.2, and it is an additional failure at the IESG level 
which you are describing.

>- when a WG submits a document to an AD, that AD has to vote 'yes' on the
>document in order to get the document on IESG's ballot.  what happens if that
>AD has problems with the document?  he iterates with the WG.  what happens if
>the AD feels that the document is so bad that it can't easily be fixed?  the
>process stalls.  the AD needs more time to come up with a set of fixes that
>the WG will agree to before the AD can endorse it, and the AD's time is in
>short supply.  the longer the document(s) in question, the more time it takes.

I agree that this happens and is a problem, but again, at root it 
sounds like a 2.2 problem.

With this, and your previous comment, perhaps these two results (the 
IESG response) of the 2.2 problem need to be described fully in 
appendix A. However, I do think the document gets at the root cause 
of the problem and identifies what we need to address when we get to 
the solution phase.

pr
-- 
Pete Resnick <mailto:presnick at qualcomm.com>
QUALCOMM Incorporated - Direct phone: (858)651-4478, Fax: (858)651-1102