Is this really where we want to go? (was: Re: Selecting leadership, take 2)

John C Klensin john-ietf@jck.com
Fri, 06 Dec 2002 21:38:33 -0500


Harald,

Sorry this is so late in the game, but I've been thinking about 
where this "let's form a WG to do problem definition" approach 
seems to be taking us in the context of a number of off-list 
interactions, which were reinforced by part of Marshall's 
comments earlier today.

Summary for those trying to figure out whether to read this 
note: I have concluded that the whole idea of creating a problem 
definition WG is wrong: probably a waste of time and certainly a 
distraction from the way in which we should be focusing our 
efforts.  I am going to try to explain why, then propose an 
alternative.

I suggest that there are some things the IETF historically does 
rather poorly.  This list isn't of problems we need to solve, 
but of things we need to recognize and work around, lest they 
swamp us.  Certainly it is unlikely that any change in either 
actors or processes will make us better at these things 
(although some might make us better at declining to try to do 
them).  For example:

	* We have discovered that Working Groups are rarely a
	good way to get from <interesting-buzzword> to a serious
	set of definitions and requirements.  Working groups
	charged with coming up with requirements in ill-defined
	areas often go on rathole tours and even more often take
	much longer than anyone expects to produce anything.
	And the results are rarely suitable for real evaluation
	and examination, as evidenced by the number of
	subsequent "protocol" WGs who end up ignoring or
	rejecting a significant fraction of the "requirements".
	
	* We have discovered that Working Groups are rarely good
	at doing real "blank sheet" design.  Either some design
	shows up from outside the WG and the WG endorses it,
	perhaps with some tuning, or the WG ends up going around
	in circles until it either produces garbage (which gets
	approved because everyone but a few advocates are
	exhausted and no one has the heart to say "no" because
	so much time and energy has been invested.
	
	* And we have discovered that we don't do "thinking
	about process" WGs very well.  Given a clear problem and
	a forced schedule, such as was the case with the
	original Poised group, we can often figure out something
	decent.  But, with a looser schedule and a vague problem
	definition, we wallow, nit-pick, practice becoming
	amateur lawyers, and spend huge amounts of time on
	things that are unlikely to make very much difference.
	And we often spend even more time looking at cases that
	haven't occurred and are unlikely to do so.  The more we
	wander in those weeds, the more we devise "solutions"
	that aren't testable -- we may have code, but we don't
	have a clue as to whether or not it actually runs.  The
	last couple of years of Poisson were a fairly good
	example, but we have many more specific ones.

I'm not trying to start a discussion here about whether we 
should avoid attempts that fall into those categories at all, or 
how they should be constrained and managed.  I just want to see 
if we can get agreement that these are areas in which we don't 
have a good success record... indeed, that we have a significant 
and regular record of failures.

So, what are we proposing to do?

We are going to create a WG whose purpose is to figure out what 
the problem is, so that we can create a WG to design a solution 
and then, either in that WG or in some other one, we can use a 
WG to start writing procedures.  Two things are wrong with this: 
each of the steps falls squarely into our areas of incompetence 
as identified above and, given the tendency of things in those 
areas to drag out, the solutions are likely to be outdated by 
the time we discover them.

If we think there is a problem, or some set of problems, that 
are important enough that we would like to see them dealt with 
soon  (e.g., certainly within the next calendar year and 
preferably before a new IESG is seated), that strategy borders 
on the silly.  I know it is not Harald's intent, but the effect 
would almost certainly be to postpone any real improvements -- 
even the obvious ones -- on the grounds that the WG is thinking 
about the grand issues.  Good way to do philosophizing; terrible 
way to do engineering.  Philosophizing isn't supposed to be our 
goal around here.

Radical suggestion:

Forget the "big problem", "big picture", and general surveys of 
what is wrong with the IETF and stop feeding whatever lives at 
the bottom of the rathole.  Assume that the current general 
structure/ framework is good enough and that what we need to do, 
at most, is some tuning, not a "blow it up and start over" 
approach. (Skeptics should remember that, before the Kobe affair 
and the Boston Tea Party, we had an IAB, an IESG made up of Area 
Directors, and Working Groups.  And, when we got finished, we 
had an IAB, an IESG made up of Area Directors, and Working 
Groups.  Yes, a lot of roles changed, and things got retuned, 
but no large changes in the structure on which things are hung.)

Instead, note that the PACT document, my response to Geoff and 
Marshall, Mark's note, and several less comprehensive (in the 
sense of shorter and with fewer individual points) postings can 
be decomposed into a series of statement sets of the form of:

	* X appears to me/us to be a problem
	
	* Because of X, the following bad things happen: A, B,
	C,...
	
	* One could do Y, which would change or modify X so that
	those things would not happen

Now, it seems to me that, whether we do it as a WG or simply on 
this list, we ought to be able to look at each of those sets in 
turn (and any others that can be proposed).  Stated that way, 
serious discussions about each element are possible: Is X really 
enough of a problem to bother with?  Can we agree that A, B, and 
C are actually consequences of X, or might they result from 
something else that needs to be looked at?  Does Y really solve 
X, or does it just attack a subset of A, B, and C?  And are 
there side-effects to Y that are worse than X itself?  Is it 
safe to deploy Y by itself, or are there interactions with other 
things that should be considered first (and what are they)?

That is a case-by-case analysis problem with well-understood 
types of questions.  It is fairly close to what we try to do 
with network protocols being proposed for standardization.  If 
we care, we can usually deal with those sorts of problems fairly 
effectively.  And it is very different from going off into the 
weeds hoping that we can find a crisp problem definition out 
there somewhere and do it in finite time.

So I would suggest that:

(1) We either start that process here, or spin up a WG whose 
function it is to start working through the proposals.

(2) We adopt approximately the same rule toward proposals to be 
considered that we usually adopt in well-managed WGs, i.e., 
nothing gets discussed unless there is a clear written statement 
of it, and, in this case, that statement contains the "problem - 
explanation of consequences - proposed solution and its 
implications" statements implied above.

(3) That process, and the proposals/statements, try to clearly 
distinguish among

	* issues that can be addressed by different attitudes
	among, e.g., IESG members,
	
	* issues that can be addressed by additional guidance to
	the IESG, and
	
	* issues that require changes in how we are organized
	and the ways in which we do things.

When we can reach agreement and there don't seem to be complex 
interactions, the first category should be treated by the 
Nomcom, and the second by the IESG, as real-time input with the 
expectation that they will react in real time.  The third 
obviously will take somewhat longer, but perhaps we can start 
seeing the beginnings of changes and results by the time we see 
each other in San Francisco, rather than getting there with just 
the hope of a report at a plenary to be followed by more 
discussion and then by another WG.

    john