Is this really where we want to go? (was: Re: Selecting leadership, take 2)

[Theodore Ts'o]

On Sat, Dec 07, 2002 at 05:50:03AM +0100, Harald Tveit Alvestrand wrote:
> >Instead, note that the PACT document, my response to Geoff and Marshall,
> >Mark's note, and several less comprehensive (in the sense of shorter and
> >with fewer individual points) postings can be decomposed into a series of
> >statement sets of the form of:
> >
> >	* X appears to me/us to be a problem
> >	
> >	* Because of X, the following bad things happen: A, B,
> >	C,...
> >	
> >	* One could do Y, which would change or modify X so that
> >	those things would not happen
> >
> >Now, it seems to me that, whether we do it as a WG or simply on this
> >list, we ought to be able to look at each of those sets in turn (and any
> >others that can be proposed).  Stated that way, serious discussions about
> >each element are possible: Is X really enough of a problem to bother
> >with?  Can we agree that A, B, and C are actually consequences of X, or
> >might they result from something else that needs to be looked at?  Does Y
> >really solve X, or does it just attack a subset of A, B, and C?  And are
> >there side-effects to Y that are worse than X itself?  Is it safe to
> >deploy Y by itself, or are there interactions with other things that
> >should be considered first (and what are they)?
> 
> If you remove Y from the equations above, don't we have a fairly good start 
> at a problem statement?

It's not just a matter of removing Y.  There's a very important step
between creating a problem statement, and then chartering one or more
working groups to solve the problem statement, and that's the
validation of the problem statement.

It's easy enough to brain storm a bunch of problems that purport to
afflict the IETF.  But after that, we need to pass them through a
couple of filters:

*) Is this a major problem, or a minor one?  (Answering this question
can be complicated, because a few minor problems might interact
synergistically)

*) Does the problem have a solution at all?

*) What is the cost of trying to solve the problem?  Is the cure worse
than the disease?  (And that means we have to know what Y is in order
to evaluate the proposed problem.)

It's important, I think, to do a very thorough and careful vetting of
a purported problem before chartering a working group to solve it,
because once a working group is chartered to solve a particular
problem, it's very easy for that wg to get so focused on the task at
hand that they neglect the question of whether or not it should be
done in the first place.  And for a protocol, it's relatively harmless
(besides wasting everyone's time), if no one uses the protocol which
the working group developed, if it turns out to be worthless ----
however, in the case of a process-based working group, it's going to
be politically very difficult not to use the output of the working
group.

So we had better be sure before we set a wg off to solve a problem
that it's one where the costs of the solution are worth the ultimate
benefit.  For example, if we tell the working group, "the problem is
that standards take a long time", we may end up with a solution which
says, "throw out quality controls, and just blindly approve everything
that a draft author and 2 or 3 others want".  That would certainly
solve the problem of standrards taking a long time to publish, but the
side effects in terms of technical quality of the standards and the
resulting hit to the reputation of the IETF would probably make that
solution not worthwhile.

Presumably, one would think that the example I gave is so obvious that
no one would think try to publish an I-D that would propose that.  But
even if they didn't, I suspect there are plenty of other more subtle
cases where the side effects are not so obvious.

As a result, I would be very scared of a process where there wasn't
sufficient vetting between the problem statement, and chartering the
work group to solve the problem, and if there wasn't sufficient
vetting between a work group producing a solution, and the IETF
deciding to adopt it.  Merely presenting the idea at a plenery and
asking for a "hum" doesn't seem to me to be nearly thoghtful enough.
(I can understand not wanting to have the IESG vet the output of the
problem-solving working group, given the perception by some that "it's
all the IESG's fault".  However, *someone* needs to vet the output
carefully.  We really don't want to be making fundamental changes to
the way we do business with less oversight than we apply to normal
protocol documents.)

> I don't want to stop improving things while this goes on.
> We'll always be working with things like tracking improvements, chasing old 
> stuff down, resolving comments speedily, adjusting attitutes, working on 
> training and so on.
> 
> as you say, there are lots of problems that do NOT need any changes to the 
> process to solve.
> 
> but I don't think those are the underlying problems that led us to where we 
> are now. And I'd like to have those talked about.

I don't know what we will find as we start trying to uncover the core
problems.  But we should be open to the possibility that there may
*not* be any big, huge, horrible, fundmental, underlying problems.   

While there may be, it is at least as likely that there might be
instead a some number of smaller problems, whose negative effects
combine multiplicatively.  If that's the case, it may be that as the
IESG and the secretariat continue to deploy incremental fixes, these
"small fixes" might make the problem go away, or be significantly
reduced in size, during the life of the problem statement wg.  We need
to be open to that.

						- Ted