The "late surprise" problem

[John C Klensin]

Brian (and Dave),

I think this is a very interesting idea.  I think your
analysis and presentation of it also points out one of our
core problems, one that is perhaps another aspect of my
"systems analysis" observation.  At one level, your proposal
should help with some of that problem, especially if it is
enhanced by Harald's "color code" idea.  But it also
illustrates the problem, so let me pick on you a bit.

Our tendency is to want to say "let's identify a piece of the
problem that can be solved, and propose a solution to it".
You have done that and, IMO, done that quite cleverly.  But we
are often not taking the third and fourth steps in that
process, which I think are often critical.   Those steps are
figuring out:

	* What side-effects the solution would cause on other
	aspects of our work, protocols, etc.?
	
	* Is the solution (or the problem being solved)
	important enough to justify accepting those
	side-effects.   Or can it be reengineered to reduce
	its negative impact?

We keep skirting around parts of this.   Stuffing "foo
considerations" into documents can be seen an attempt to look
for, and document, particular classes of side-effects.  An "at
least one clue of each flavor" is another approach, as is
having the whole IETF look at every document.   But, even when
we do side-effect identification, we are often poor and really
evaluating those effects and the associated tradeoffs (this is
an important impact of the "you chartered our WG, and this is
our output so you have to approve it" attitude (and why I'm so
concerned about default approval of anything a WG produces).

To illustrate what I'm talking about, let's take your proposal
as a given and try to look at those other steps.

We have heard many comments that the IETF contains a core
group who really make the decisions, and that it is far too
hard for new people to get into it.  If one identifies that
group as the IESG and the IAB, at least the group is small,
and we have clearly-defined procedures for getting in.  The
hangers-on to the core are larger and more amorphous, which
may be good.   Separately (maybe) we have a leadership
development program: we are not as good as we should be at
bringing new people into the community and developing them
sufficiently to expand the pool of potential WG Chairs, IESG
members, etc.

Now suppose that we codify the core by formally designating an
elite at a size around an order of magnitude above that of the
IESG and one or two orders of magnitude smaller than the IETF
community.  A ceiling on membership makes it very difficult
for anyone else to get in --they mostly have to wait for
someone to die, probably by being observed to have retired in
place.

So, for example,

	(1) We build a large, institutionalized,
	formal-membership, cabal.  And we claim we don't like
	cabals.  For example, one would imagine that a Nomcom
	would have a lot of trouble selecting people who were
	not part of the SIR group (plus WG chairs and
	incumbents).  Maybe that is a good thing, but it would
	be good to understand the possibility.
	
	(2) Unless the "reviewer" process also includes a good
	deal of mentoring, informative feedback and tutorials,
	it could make the leadership development problem worse
	(I assume everyone can fill in the logic about that).

Now, those sorts of considerations don't make the proposal any
worse.   It may be useful enough to ignore the impacts.  But I
believe that looking at these second-order effects is
important to understanding the proposal itself, the things we
need to look for as we try to analyze it, and what other
things we might need to adjust if we accept it.  And we do too
little of those things, IMO.

      john