The need for smaller protocol specifications

[Patrik Fältström]

The overall problem I think is that in general people in the IETF don't 
spend enough time reviewing documents and architectures. One small 
group of people which soooo much want something done, something which 
makes sense from their perspective write a draft, and "push it through 
the system". The only real pushing today which is needed is the 
creation of a working group.

When the working group is created, the pushing is within the working 
group. Either we have an active wg chair which ensure review and proper 
document OR the small group keep pushing.

When the wg is "ready" with the document (either the document is really 
ready, or the small group has pushed enough to make the wg chair tired) 
the document end up with IESG.

What are the different paths to here? (from my perspective after being 
5 years AD, 1998-2003):

(1) The document is really good, as it should.

(2) The small group of active people is still active. The wg chair was 
tired and no one in the wg except the small group has reviewed the 
document. The document in reality because of this comes to the IESG 
from the small group without any other review.

(3) The small group ended up being so tired after pushing for the wg 
and pushing the wg chair that they faded away. The wg chair (maybe not 
even the same as from the beginning of the wg) has more with whip than 
carrot managed to get some other doc editor to finally produce the 
document. The document is quite good quality, but, the original idea is 
lost, and possibly the final document is not the bright idea as 
originally (and it is normally _very_ late).

(4) The document never reach the IESG at all.

During my years, I saw fewer and fewer documents in the (1) category. 
The only category where IESG can "rubberstamp" an I-D. As area director 
I didn't want anything else than just rubberstamp something.

More and more drafts ended up in category (3). A few in (2). As (2) and 
(3) has to be treated so differently, it is hard as AD to find a good 
mechanism for how to work efficiently. Also, an increase of documents 
in the (2) and (3) category increases the workload of the AD.

I feel Charlie talks about more category (2) documents be classified by 
the AD as (1). This means the AD should "trust" the wg doing a good 
job. This is exactly what the AD want as well, BUT, part from being the 
last line of defence against crap, the AD is also the one which decides 
whether proper review has actually happened.

Personally, I looked for a few things to make the decision whether a 
document was (1) or (2). Simple things which is in the I-D Nits list. 
They include:

  - Do grep for the following strings "UTF-8", "Unicode", "Character", 
"ASII".
    If any of these are included, are they described correctly?
  - Check if normal domain names are used, or "example.com" style.
  etc...

If the document fail on any of the above, the document is clearly a 
category (2) document. However I wanted it to be a category (1).

Category (3) is the hardest though. A document finally arrives at the 
IESG. If the AD find some bugs, a message is sent back to the wg chair 
(and in some cases the wg). Nothing. A few weeks later, one might 
remember to ping the wg which in reality is in what I would call zombie 
state. Finally something comes back, and then the AD doesn't remember 
what happened, and the mental/thinking-process needs to restart. VERY 
expensive.


When reading this list, I find _all_ people seems to think we should 
have more documents move from category (2) -> (1), and possibly more 
(3) -> (4).

That will definitely decrease the workload of the AD, we get rid of 
boring crap (which possibly will not be used on the Internet anyway). 
The wg will get more power as the number of category (1) documents 
increase.

BUT, it will be hard to get there.

The reason why I as AD felt more documents ended up in (2) or (3) 
include:

  - I found more and more documents with bugs which could have been
    fixed by doc editor just by reading I-D Nits (see above)
  - I almost _never_ got any reaction what so ever (positive or negative)
    on IETF Last Call

In this situation, and being the last line of defence, I feel I have to 
do a more detailed review. I feel I can not, however I would like to, 
trust the wg.

At the same time, this leads to the AD being questioned doing 
micro-management, requiring too much from the document, etc etc which I 
see between the lines in what many have said.

And, I really understand this complaint. I really do. Really really 
really!


I feel the meta-question is how to start turning the wheel the other 
way around. How do we make sure the wg produce documents which can be 
rubberstamped by the AD, documents which are not crap? How do we make 
wg's produce better documents? How do we make sure Last Call is 
happening?

I state it this way because I don't think the problem is what level of 
quality IETF in the current process require. I think the problem is 
really that we don't know where it is evaluated whether the document 
has reached this level. Whatever the level is.

Many many people agree it is better if this review happen early in the 
process. How? How do we know it has happened? I know the only way to 
convince an AD that it has happened is to give documents to IESG which 
does NOT fail any of the things in the I-D nits document. Claiming it 
has got review, and then failing I-D nits is not good. At the same 
time, having an AD late in the process require changes is not good 
either? It polarises things. It ends up  being a "we" and "them". Not 
good.

We have the timeline in the wg charters. Maybe what we need is an 
objective way of forcing documents through the process. We create real 
steps the doc have to go through, and each step has a deadline. If the 
deadline is not met -- start from the beginning again. The deadlines 
are set by the wg like today. That way we at least remove all category 
(3) documents from the IESG load, but we might also be able to "force" 
the wg to do more proper review? It might be easier for both a wg chair 
and AD to "just say no" and then spend time on things which need real 
help, like synchronisation across areas _during_ the creation of a 
document and not after.

I don't know.

But, I definitely think we need to talk more carefully about where the 
review is done, and how we make sure it is done. As AD, I found it very 
very hard in problematic wg's to both review process and document 
quality. Very often these two roles are fighting each other.

     paf