Staying on Track (Re: Documenting pilots (RE:
pausable explanation for the Document Series))
John C Klensin
john-ietf at jck.com
Sat Jun 7 11:01:34 CEST 2003
Dave,
I need to _slightly_ support Melinda's position on this. I
certainly wouldn't have used words like "preemptive" or
"uncooperative". But...
In a cultural problem that goes back long before we, as you
observed elsewhere, started putting more focus on static
description than on demonstrated interoperability (and, IMO,
blew it on both), we have had a habit of proposing, documenting,
and sometimes even trying "experiments" which lack evaluation
criteria... or any way, other than force of personality, to
really determine whether they "worked". Nor do we have real
procedures for documenting what we have learned.
In that regard, SIRS has been, I think, better than the average.
But it is a handy example, so let's look at it, with the
understanding that I'm deliberately broadening the example a bit
to illustrate a problem with the way we often do business ...
some of that example may not actually apply to SIRS.
It establishes goals as:
> The procedure described in this document is intended to
> solve, or palliate, a number of related problems that
> have been observed in the IETF process [PROBLEM]:
>
> * overload of the IESG (leading to delay)
>
> * perception that authority and influence
> in the IETF are concentrated in too few
> hands
>
> * submission of documents to the IESG that
> still have significant problems (leading
> to delay)
>
> * failure to detect fundamental problems
> and Internet- wide issues at an early
> stage
Certainly worthwhile. But, quoting from your note, "...started
to
offer itself for public use, and will prove useful -- or not --
on its merits. How are those merits to be evaluated so as to
determine whether this is a success? The document then goes on
to say:
> However,
> it is likely that drafts with at least three positive
> reviews from SIRs in different areas will experience
> much shorter IESG review cycles than drafts with fewer
> positive reviews.
[...]
> In all likelihood, Drafts without reviews will get
> worse IESG response time than today, whereas Drafts
> with reviews will be processed much more rapidly,
> especially as the IESG's confidence in the SIR
> procedure increases.
Plausible hypotheses indeed. But, for a given document coming
out of a given WG, we have no practical and objective way to
accurately estimate how long it would have taken to get through
the IESG without SIRS support, so it is not possible to
objectively know whether SIRS sped thing up.
I don't think it likely, but a counter-hypothesis is that the
IESG, upon receiving a document with three or five SIRS signoffs
but disagreeing anyway, would decide that it needed to give more
extensive explanations to those senior people, or discuss issues
with them even before going back to discuss them with the WG.
That would probably positively leverage quality, but would have
a negative effect on speed.
And, of course, trying to guess at whether SIRS improved quality
in comparison to what the IESG would have done, to/with a given
document without it. is even more difficult.
So, you collect some reviewers, and some WGs choose to use them.
For some of those WGs --perhaps the ones who were in good shape
anyway but that, too, is pretty subjective-- the reviews help
the IESG expedite the process. For others, they don't appear
to. Do we then declare the experiment a success and try to make
it mandatory? Does anyone who doesn't think it has proven
itself a success get to speak up without being overwhelmed in
rhetoric and forceful assertions? Do we infer that ADs who
don't quickly sign off on SIRS-endorsed documents are
obstructionist nit-pickers and need to be fired? And so on.
As I hope my recent notes make clear, I'm not in favor of trying
to invent procedures to alleviate any of the above, or of
delaying things until such procedures reach some extraordinary
consensus. But I'd be much happier about the idea --even as an
experiment-- had the IESG been willing to stand up and say
"fascinating idea, lets try it". And it would have been even
better, at least from an "experimental" context, had the IESG
chosen to say "It would be good to really do this as an
experiment and therefore to be able to make some comparisons.
So let's look at the following mix of areas, or specific groups
of WGs, first, leave the others with status quo, and see how
things work out.
I'm disappointed that type of response from the IESG hasn't
happened. Maybe they don't think they have been asked. And
the latter is, of course, what my recent notes have been about:
if we need a formal document and request to publish it as an RFC
and/or a formal WG call for consensus and response, in order to
get an IESG response to a "we think this would be worth trying,
do you have reactions or suggestions/requirements about how to
try it", then, IMO, we are in deep trouble. But, going ahead
with something on the grounds that it is an experiment, without
objective evaluation criteria or any real possibility of them,
and without IESG comment, input, or participation... That seems
to me to be an opportunity for some future demagogue, with some
other experiment, to try something, claim it succeeded, and then
insist --loudly and by whispering campaign-- it was a success
and that IESG failure to immediately adopt it indicates that all
of those rascals should be thrown out and the procedural
structure blown up (and replaced with one of the demagogue's
liking) to mark their way.
john
--On Friday, 06 June, 2003 16:15 -0700 Dave Crocker
<dhc at dcrocker.net> wrote:
> Melinda,
>
> MS> I'm not crazy about things like what's going on with the
> MS> SIRS proposal (going off and collecting volunteers without
> MS> real agreement from the cohort that this is a good thing to
> MS> do) because it feels pre-emptive and unco-operative,
>
>
> Oh boy, do we have different views about such efforts.
>
> Let's be clear about the nature of this sort of "preemptive"
> and "uncooperative" effort:
>
> 1. At the last IETF meeting, Brian came up with an idea,
> chatted it up, got good reactions and suggestions, recruited
> someone to help with the writing a few paragraphs on it and
> then circulated the idea informally.
>
> 2. The writeup was chatted up, got good suggestions and
> reactions, and turned into an I-D.
>
> 3. The I-D got chatted up, got good suggestions and reactions,
> and turned into an experiment.
>
> 4. The experiment has, so far, gotten volunteers, started to
> offer itself for public use, and will prove useful -- or not
> -- on its merits. It has no force of law, or anything other
> than basic credibility.
>
> It also has not had months and months of delay.
>
> So, in fact, this is about as far as you can get from being
> preemptive or uncooperative.
>
> On the other hand, yes, it is also very far from relying on
> bureaucratic approval.
>
> Rather than being worried that it has occurred, perhaps we
> should be worried that more such efforts have not?
>
> d/
> --
> Dave Crocker <mailto:dcrocker at brandenburg.com>
> Brandenburg InternetWorking <http://www.brandenburg.com>
> Sunnyvale, CA USA <tel:+1.408.246.8253>,
> <fax:+1.866.358.5301>
>
More information about the Problem-statement
mailing list