The "late surprise" problem

Keith Moore moore at cs.utk.edu
Fri Mar 28 07:56:49 CET 2003


> > software development is not representative of engineering as a whole, 
> 
> true, but that shouldn't keep us from considering what we've learnt there

I don't know why we should consider the lessons from software development
more applicable than lessons from other branches of engineering.

> The *major* reason for iterative development being inappropriate in some
> diciplines is that the cost and time of implementation overshadows the
> cost and time of design. Trying to iteratively build the Golden Gate
> bridge comes to mind as an example.  But even in this case, one might
> build models and do stress and cost calculations on more than one design
> in parallel, before deciding on the final one to implement.

the bridge analogy is a good one.  in the case of protocols it's important
to consider not only cost of implementation but also cost of (re)deployment
or fixing the protocol when the first design doesn't work well. 

actually I would have few problems with most IETF working groups doing designs
in parallel and testing them against some criteria, or with doing iterative
design and prototyping before producing a product.  the problem is that most
working groups don't try several different designs in parallel and declare the
first prototype ready for deployment without any testing.  it's really hard to
fix things once they're deployed. iterative refinement has the general problem
of hill climbing strategies - they are fairly efffective at finding local
maxima  but not effective at finding global maxima.  so you need to try
multiple designs to avoid being stuck with a very sub-optimal local minimum.

as for how this applies to improving IETF operation - for the most part we
don't have the luxury of trying several designs in parallel.  so incremental
refinement misses the opportunity to find other than the local maximum -
and my belief is that we're already fairly near the local maximum.
(working groups are an exception - we could improve WG operation by allowing 
different WGs to try new things, comparing how well these WGs operate, and
picking the techniques that work best)


More information about the Problem-statement mailing list