what is a problem

[Patrik Fältström]

On tisdag, nov 26, 2002, at 00:35 Europe/Stockholm, John C Klensin 
wrote:

>> This state is something only(!) Standard Track Documents or
>> BCP go to. Writeup of a discuss is only needed when a ballot
>> is issued. Not on experimental or informational documents.
>
> I'm glad to hear that.  If I've got an individual submission
> informational document in that state, what did you say I do
> about it?   Given that the IETF Chair is aware of the problem
> (and, indeed, if the tracker is to be believed, is the
> token-holder/ obstruction), I hope the answer isn't "appeal".

No, if an individual submission is there, that is I would claim a 
mistake in the entering of data into the tracker. If I were you I would 
first complain at the token-holding AD and IESG-Secretary. The 
IESG-Secretary email address is today a ticket system, which means you 
get a ticket opened.

At the end of the day, of course _any_ administrative errors is an 
appeal case, but at the moment, when so many things changes at the same 
time as we try to do the day-job, I really hope people first try to 
"just" let IESG/IESG-Secretary know about errors like this.

You see, there _is_ an ability for an AD to request a last call for 
non-standards-track/non-bcp document. I personally have used it about 
once a year, when a request for informational/experimental is _not_ 
from RFC-Editor, and _not_ from a wg, and I do not feel like if I am to 
make the decision myself about it. In those cases, which are 
exceptions, it has sometimes been mistakes done and a "ballot" is 
created. This just because the creation of ballot has been 
semi-automatic after last call.

> To repeat (briefly) a point I've tried to raise elsewhere, under
> what authority does the IESG claim the right to take "discuss"
> positions on Informational document that do not specify
> protocols, address issues that are included in WG Charters, or
> suggest courses of action that could be harmful?  I can't speak
> for others, but the reason I've been inquiring about an IESG
> Charter is to get the IESG to identify the range and limits of
> its authority and responsibility so that the community can
> determine whether or not it agrees.

My view is that IESG do very very little review of the Informational 
documents. But, this is of course one of the areas where discussions 
like this will help making things more clear.

I see a couple of different Informational documents. Let me check with 
you if you see a difference between them:

  - Individual submissions which the individual sent to the RFC-Editor
  - Individual submissions which the individual sent to the IESG
  - Wg submissions to the IESG

What I do myself is to literally do a "grep" for things like "URI 
scheme", "Unicode", "character", "charset", "IANA", "Security 
consideration" etc. This because I try to discover if the document is 
actually a registration (or should be a registration template) for an 
IANA parameter which I feel is within my responsibility to keep track 
of.

I also browse through the document to see whether it overlaps with some 
wg, and if it does, I ask the editor that they should send the document 
to the wg mailing list (and I inform the wg chair). Same thing with 
email related issues (-> rfc822-list) and URI things 
(discuss@apps.ietf.org and uri@w3c.org).

Nothing more.

I try to be extremely careful with not doing a "review".

Of course, if I do "too much", I need to be told by external parties 
(like you). That doesn't have to mean appeal. Just "hrm...Patrik, what 
is this".

As I said before, the question asked to IESG is "Does anyone have any 
problems with this document being published as Informational". Silence 
means the documents "passes".

>> Now, the reason why things are stuck there has to do with the
>> overload of individual AD's which John pointed out during the
>> plenary, and his note to the IETF list.
>
> But one of the ways to reduce overload is to reduce the number
> of things the IESG starts trying to review and comment on at a
> detail level.  It seems to me that deciding, within the IESG,
> that you must, or even can, start quibbling about textual or
> editorial points in individual informational documents just adds
> to the overload and that you (collectively) should get no
> sympathy at all in cases where you take on work that no one has
> asked of you and that is not necessary to the integrity of the
> standards process.

I agree completely with this. IESG should not do that. I can not see we 
have said we should quibble about it either. if we do "too much", then 
that has to be corrected.

As we are verifying whether an _Individual_ submission is possibly 
overlapping with what a wg do, I feel sometimes we have a gray-zone, 
where the only thing I can do is to recommend the editor to be in 
contact with the wg, or, to let the wg chair know the document is on 
it's way pass the IESG and I want to get feedback (this is between 
Agenda is created, and the telechat) really fast whether the wg chair 
think there is overlap.

>> Something we have been talking about in the IESG is that with
>> the help of the tracker, we have the ability to send
>> notifications when certain events happen. Events in the form
>> of "new I-D did appear in the repository" or "things have been
>> stuck at this state for too long time".
>>
>> We don't have this mechanism yet, so we individual AD's rely
>> on individuals as ourselves and now also you other IETF people
>> to let us know.
>
> Does this imply that you are telling the community that, when
> something appears to be stuck, we should harrass the IESG?  I
> had been assuming that you could do a better job in managing an
> overloaded situation if the other 2000 +/- of us weren't
> continually on your cases, complaining about what you were not
> doing, and expecting responses.

To a certain degree, yes, I think you _should_ harass us.

But, there are different degrees of this as well.

The reason why I ask you to keep your eyes open this year are many:

  - IESG try to write down not only how we should work, but also
    how we work (as Pete pointed out, the map and reality might
    differ).
  - We are trying to move data about the documents from a manual
    system at the Secretariat to the automatic database in the form
    of the Tracker.
  - The "Evaluation" IESG does is still completely manual by the
    Secretariat. And results from Evaluation is to be entered in
    the tracker manually.

At this point in time there are a number of places in the procedures 
where administrative mistakes can be made.

Let me explain in more detail:

Until some 4 weeks ago(!) the Secretariat handled all status of 
documents. They (and some AD's) used the tracker, some other AD's (like 
myself) didn't use it so much. Reason for me not to use it was simply 
that it was too slow, and a very long rtt from Sweden made it not 
usable for me.

Changes of status was done by having the AD exchange email with 
Secretariat.

Today, I claim most AD's use the tracker, and the tracker is to contain 
the authoritative information about a document. But, some things are 
missing:

1 We have no "alerts" or "timers"
2 We have no automatic "evaluation"
3 We have no automatic way for people to "submit" documents to IESG

We do work on adding these features as well, and when we have it, you 
don't need to harass us as much (I really hope, or we have a real 
problem).

Until we have this, a few things can happen (and happens):

(1) Things are stuck at a state, like "writeup needed", too long
(2) Evaluations which are handled by the secretariat is entered not in 
detail exactly the way the AD wants (either AD taking care of the 
document or the AD doing the evaluation), which include entering the 
Discuss comment
(3) People requesting a document to be moved through the IESG is lost 
due to dropped ball (people don't send to iesg-secretary@ietf.org or 
RFC-Editor but instead directly to an AD and think they are done).

And, seeing that these three things does NOT happen is something we 
need help with.

I don't know if harass is the correct word for that "help", but why not?

>> Note, things should be in "writeup needed" only a few days, or
>> maximum 2 weeks, between two telechats. (For various reasons,
>> 2 weeks is the in practice shortest timeframe IESG can act on
>> for major state changes in reality. Sometimes things can go
>> faster, but that is modulo availability of many people.)
>
> The example in question went into that status, according to the
> tracker, on 5 November.  I would feel somewhat more sympathetic
> to this situation, with informational documents (I think it is
> dangerous for standards-track ones), if "discuss" followed by a
> two-week silence implied an automatic release to the RFC Editor
> with a "no IESG comments" indication.

See above about the different kind of Informational Documents. I don't 
know the name of the document, and am on a train so I can not check 
this specific one. But, yes, that is what I think we in IESG have said 
about documents being fed from the RFC-Editor.

The only thing IESG can do, I think, let me know if you don't agree, is 
to within the specific timeout period set by the RFC-Editor is to 
explicitly request an extension in writing. And, the RFC-Editor can of 
course refuse to extend the timeout.

Reasons can include:

   - An AD which should look at the document is on vacation and need
     an extra week
   - The review period is over an IETF meeting so an extra week is
     requested (note, last calls are extra long over IETF meetings)
   - The IESG has requested a one week review by some wg, and would
     like the RFC-Editor to consider input from that wg, if any
   - The IESG would like a "normal" expert review is needed because
     the document involves registration of IANA parameters, and IESG
     would like to know what the expert review result is

Do you agree?

But, that should definitely be a request _in_writing_ which means that 
the request itself should be recorded in the tracker.

Black hole should not be acceptable.

>>> And remember that,
>>> unless the IESG has quietly expanded its role (again), the
>>> only possible comments on this type of document are supposed
>>> to be "ok to publish if the RFC Editor wants to do that",
>>> "end-run around the foo WG's efforts/charter or around
>>> standards-track RFC ZZZZ", or "damaging if deployed".
>>
>> Here we have a different issue. Here we have a draft which is
>> supposed to be Informational or Experimental. No evaluation is
>> used. The question to IESG is "Does anyone have any problems
>> with this being published as Informational" (or equivalent).
>
> That, to me, is a correct question.  The answer, if "yes" (i.e.,
> there are problems) should be substantive and based in one of a
> small number of principles and not ever editorial unless it is a
> WG product that qualifies or elaborates on standards-track
> materials.

We completely understand each other here.

>> What we _know_ we have problems with is if an individual IESG
>> member have issues. My suggestion, now with the new tracker
>> etc, is that we should handle these issues exactly like the
>> Discuss for BCP and standards track, BUT, modulo exactly what
>> you John state. That I-D's that come from RFC-Editor is to be
>> treated differently.
>
> But, apparently due to IESG instructions, I-Ds don't come from
> the RFC Editor.

This is wrong. People can request a publication of an RFC by contacting 
the RFC-Editor. The RFC-Editor will issue a "timeout" together with 
notification to the IESG to give IESG ability to send input.

The problem which RFC-Editor and IESG discusses is how to ensure this 
timeout is in reality working.

I claim we have no disagreements here between RFC-Editor and IESG. The 
truth is that we need to write down _exactly_ how it is to work.

And, believe me, the creation of the tracker has helped documenting the 
process. What we do now is to look over the "timers", and that will 
even more make the process more clear.

> If there is any desire to get a document
> processed and published in finite time, it has to go to the
> IESG.  If there is some other procedure that is supposed to
> work, I continue to hope that the IESG will document it.

Reason for this is because (a) IESG only have so much time during a 
telechat and does't always get though the whole agenda (b) Timeouts are 
in reality not implemented.

This has to be fixed. Especially the timeouts. I acknowledge this. The 
IESG is aware of it. And it _WILL_ be implemented in the tracker.

First thing which is implemented though is the automatic ballot which 
also will reduce the amount of email back and fourth between AD's and 
Secretariat. A "discuss" today involves the AD which is responsible for 
the document, the AD which rises and issue, and the IESG Secretariat 
which is to record the result of the discussion. This is stupid and 
cost too much (in time).

>> The notes that can be needed are:
>>
>> (1) The issues themselves (of course)
>> (2) An IESG Note which IESG want to be added
>> (3) A Do Not Publish statement from IESG
>
> I even question "the issues themselves" unless the IESG is
> identifying a conflict or end-run or source of confusion with
> IETF-approved activities.  Nothing in procedures established and
> reviewed by the community give an IESG member any more input
> into the ideas of an individual submission than any other member
> of the community at the time the document is exposed as an I-D.

With "issues" I only meant things like the ones listed above.

We are not in disagreement here.

> Once again, if you think you need that responsibility and
> authority, please --whether by a charter or some statement that
> can be reviewed-- ask the community for it.   Assuming it in
> secret is just not a good idea.

I agree.

>>> Instead, we have a procedure that apparently permits IESG
>>> members to drop things into black holes.
>>
>> Because we have no mechanism to keep track of the timers, yes.
>
> And because you have, apparently, no mechanism to prevent
> experimental and informational documents from being tied up
> using mechanisms that you have identified as only appropriate
> for standards-track ones.

Sort of, yes. My view is that because we don't have timers, because we 
don't have the automatic release, and because we don't have a proper 
recording mechanism of issues (once again, only the formal ones 
allowed) -- the _result_ of an AD say "I have issues" is what you say.

I claim, I might be wrong, that by more administrative help we can get 
rid of this problem. The administrative help is the mechanism you ask 
for.

Until we have it, the only help we can get is by having you all 
"harassing" us.

Once again, it is _NOT_ the case things should be blocked because of 
non-formal issues.

>> Let me summarize:
>>
>> - For standard track and bcp, we _do_ know who we are waiting
>> for when waiting for writeup
>>
>> - For Informational and Experimental, we have been talking
>> about using exactly the same mechanism as the above
>
> My personal recommendation is that, before doing this, you
> carefully delineate under  exactly what conditions objections/
> "discuss" positions can occur.  And, if those conditions exceed
> the review scope specified by RFC 2026 section 4.2.1 (for
> Experimental), 4.2.2 (for Informational), and 4.2.3, they need
> to be written down and community approval obtained.   I note
> that the relevant text in 4.2.1 and 4.2.2 is "subject only to
> editorial considerations and to verification that there has been
> adequate coordination with the standards process" and that
> "editorial considerations" fall clearly (in that document) under
> RFC Editor control.

I claim we try to follow these rules.

>> - We _do_ have problems with timers, but this is not the same
>> thing as having AD's intentionally black-holing a document
>
> I ultimately don't care whether something vanishes into a black
> hole because of intentional acts or just because of overload.
> However, if the latter is the problem (as I believe it usually
> is), I think we should be looking for ways to put less load on
> the IESG, rather than having the IESG make up more ways to add
> to its load, especially without consulting the community.

I completely agree with this.

The way I think we should go forward, which is what I feel IESG have 
consensus on (and if you don't agree, that should be identified), is 
that we take incremental steps forward here.

Step 1 has been discussed for some time, and explicitly identified in 
Yokohama: OPEN IESG INTERNAL PROCESS

We have the tracker which means you all can view what is happening. 
With the help of it we have identified already some months ago that we 
need timers and automatic ballots to make sure things happen timely, 
are not lost etc. These two things will happen.

Step 2 has also been discussed for a while in IESG, and that was 
clearly identified in Atlanta: REDUCE WORK LOAD OF IESG

This can be done in a few different ways: increase the number of IESG 
members or move things away from IESG. I think we in Atlanta found 
consensus that we should move things away from IESG. I have myself 
thought this was the correct way of going for a long while BUT, that is 
not possible without knowing someone else want to do more work.

My personal view is that more work have to be done in the wg's, and 
IESG need to "trust" wg's etc more. We have ended up in a bad spiral 
where quality of documents have went down, people don't review 
documents (because IESG do it anyway) and IESG have to spend more time 
on it.

IESG should NOT have to do more than browse through things. The more 
the IESG do in the review process, the worse. Documents should be 
reviewed in the whole IETF community, and not by the IESG. IESG should 
help ensuring there is consensus, and that things "fit together". And, 
as Marshall and others pointed out, that things "fit together" is input 
which should be sent _EARLY_ in the process. Not at the last review.

We have a few different groups which can do more work: Wg Chairs (as 
the ones ensuring the wg do good work) and "Experts" in the form of 
directorates or whatever, which the AD appoint.

I really would like to see more review not only in the wg, but also 
between wg's.

To help moving things away from IESG, IESG have now created yet another 
ID-Nits document which is more in the form of a checklist. I am sorry 
to say that as long as probably 75% of the documents coming to the IESG 
doesn't pass the semi-mechanical checking according to the nits, I 
don't know what to do.

I _really_ hope this need for offloading the IESG is not only a message 
to the IESG, but also to IETF participants and wg chairs. The better 
the review is before the document comes to IESG, the smaller amount of 
work IESG has to do. Every time we get a document which doesn't have a 
Security Consideration Section, every time we get a document where the 
MIB is not compiled or the ABNF not checked, that is more work.

I am NOT blaming wg chairs or IETF participants. The IESG is the one 
responsible for the process, and the view has been that IESG is doing 
too much, which means non-IESG do less.

But, to start turning the wheel the other way around, we (IESG) need 
help!

What step 3 is, I don't know, but I am not worried. I am sure we will 
find step 3, 4, ... as well.

Let's see Step 1 is finished (you have identified some remaining 
issues) and then let's fix Step 2 while looking for Step 3.

    paf