Is this really where we want to go? (was: Re: Selecting
leadership, take 2)
John C Klensin
john-ietf@jck.com
Fri, 06 Dec 2002 21:38:33 -0500
Harald,
Sorry this is so late in the game, but I've been thinking about
where this "let's form a WG to do problem definition" approach
seems to be taking us in the context of a number of off-list
interactions, which were reinforced by part of Marshall's
comments earlier today.
Summary for those trying to figure out whether to read this
note: I have concluded that the whole idea of creating a problem
definition WG is wrong: probably a waste of time and certainly a
distraction from the way in which we should be focusing our
efforts. I am going to try to explain why, then propose an
alternative.
I suggest that there are some things the IETF historically does
rather poorly. This list isn't of problems we need to solve,
but of things we need to recognize and work around, lest they
swamp us. Certainly it is unlikely that any change in either
actors or processes will make us better at these things
(although some might make us better at declining to try to do
them). For example:
* We have discovered that Working Groups are rarely a
good way to get from <interesting-buzzword> to a serious
set of definitions and requirements. Working groups
charged with coming up with requirements in ill-defined
areas often go on rathole tours and even more often take
much longer than anyone expects to produce anything.
And the results are rarely suitable for real evaluation
and examination, as evidenced by the number of
subsequent "protocol" WGs who end up ignoring or
rejecting a significant fraction of the "requirements".
* We have discovered that Working Groups are rarely good
at doing real "blank sheet" design. Either some design
shows up from outside the WG and the WG endorses it,
perhaps with some tuning, or the WG ends up going around
in circles until it either produces garbage (which gets
approved because everyone but a few advocates are
exhausted and no one has the heart to say "no" because
so much time and energy has been invested.
* And we have discovered that we don't do "thinking
about process" WGs very well. Given a clear problem and
a forced schedule, such as was the case with the
original Poised group, we can often figure out something
decent. But, with a looser schedule and a vague problem
definition, we wallow, nit-pick, practice becoming
amateur lawyers, and spend huge amounts of time on
things that are unlikely to make very much difference.
And we often spend even more time looking at cases that
haven't occurred and are unlikely to do so. The more we
wander in those weeds, the more we devise "solutions"
that aren't testable -- we may have code, but we don't
have a clue as to whether or not it actually runs. The
last couple of years of Poisson were a fairly good
example, but we have many more specific ones.
I'm not trying to start a discussion here about whether we
should avoid attempts that fall into those categories at all, or
how they should be constrained and managed. I just want to see
if we can get agreement that these are areas in which we don't
have a good success record... indeed, that we have a significant
and regular record of failures.
So, what are we proposing to do?
We are going to create a WG whose purpose is to figure out what
the problem is, so that we can create a WG to design a solution
and then, either in that WG or in some other one, we can use a
WG to start writing procedures. Two things are wrong with this:
each of the steps falls squarely into our areas of incompetence
as identified above and, given the tendency of things in those
areas to drag out, the solutions are likely to be outdated by
the time we discover them.
If we think there is a problem, or some set of problems, that
are important enough that we would like to see them dealt with
soon (e.g., certainly within the next calendar year and
preferably before a new IESG is seated), that strategy borders
on the silly. I know it is not Harald's intent, but the effect
would almost certainly be to postpone any real improvements --
even the obvious ones -- on the grounds that the WG is thinking
about the grand issues. Good way to do philosophizing; terrible
way to do engineering. Philosophizing isn't supposed to be our
goal around here.
Radical suggestion:
Forget the "big problem", "big picture", and general surveys of
what is wrong with the IETF and stop feeding whatever lives at
the bottom of the rathole. Assume that the current general
structure/ framework is good enough and that what we need to do,
at most, is some tuning, not a "blow it up and start over"
approach. (Skeptics should remember that, before the Kobe affair
and the Boston Tea Party, we had an IAB, an IESG made up of Area
Directors, and Working Groups. And, when we got finished, we
had an IAB, an IESG made up of Area Directors, and Working
Groups. Yes, a lot of roles changed, and things got retuned,
but no large changes in the structure on which things are hung.)
Instead, note that the PACT document, my response to Geoff and
Marshall, Mark's note, and several less comprehensive (in the
sense of shorter and with fewer individual points) postings can
be decomposed into a series of statement sets of the form of:
* X appears to me/us to be a problem
* Because of X, the following bad things happen: A, B,
C,...
* One could do Y, which would change or modify X so that
those things would not happen
Now, it seems to me that, whether we do it as a WG or simply on
this list, we ought to be able to look at each of those sets in
turn (and any others that can be proposed). Stated that way,
serious discussions about each element are possible: Is X really
enough of a problem to bother with? Can we agree that A, B, and
C are actually consequences of X, or might they result from
something else that needs to be looked at? Does Y really solve
X, or does it just attack a subset of A, B, and C? And are
there side-effects to Y that are worse than X itself? Is it
safe to deploy Y by itself, or are there interactions with other
things that should be considered first (and what are they)?
That is a case-by-case analysis problem with well-understood
types of questions. It is fairly close to what we try to do
with network protocols being proposed for standardization. If
we care, we can usually deal with those sorts of problems fairly
effectively. And it is very different from going off into the
weeds hoping that we can find a crisp problem definition out
there somewhere and do it in finite time.
So I would suggest that:
(1) We either start that process here, or spin up a WG whose
function it is to start working through the proposals.
(2) We adopt approximately the same rule toward proposals to be
considered that we usually adopt in well-managed WGs, i.e.,
nothing gets discussed unless there is a clear written statement
of it, and, in this case, that statement contains the "problem -
explanation of consequences - proposed solution and its
implications" statements implied above.
(3) That process, and the proposals/statements, try to clearly
distinguish among
* issues that can be addressed by different attitudes
among, e.g., IESG members,
* issues that can be addressed by additional guidance to
the IESG, and
* issues that require changes in how we are organized
and the ways in which we do things.
When we can reach agreement and there don't seem to be complex
interactions, the first category should be treated by the
Nomcom, and the second by the IESG, as real-time input with the
expectation that they will react in real time. The third
obviously will take somewhat longer, but perhaps we can start
seeing the beginnings of changes and results by the time we see
each other in San Francisco, rather than getting there with just
the hope of a report at a plenary to be followed by more
discussion and then by another WG.
john