Latency
Henning Schulzrinne
hgs at cs.columbia.edu
Sat Feb 1 10:34:12 CET 2003
Among many other limitations of the -00-to-RFC data is a more
macro-scale one that I don't think has been discussed: in many working
groups that are developing new protocols, the development model has
changed. (This does not tend to affect our staple working groups that
are effectively in protocol maintenance mode.)
If we take the software engineering analogy, the IETF has basically gone
from 'rapid prototyping' to the classical waterfall model
(http://c2.com/cgi/wiki?WaterFall). Almost any working group has to now
go through a much longer chartering process, fairly extensive
'requirements' and 'framework' documents, before the working group is
even allowed to talk about protocol development. This is a process
change which was (I believe) never really discussed and isn't really
documented.
Thus, I think the perception of long latency is not merely an issue of
publication delay but also that the whole lifecycle takes longer since
it is now composed of multiple such publication cycles.
There may well be reasons for this approach, but it might be helpful to
weigh its costs and benefits. The benefit is presumably better output -
the resulting work looks much different (and better) than it would have
without this effort. It would be nice to have concrete examples for that
(e.g., by asking the authors of the final spec how much the requirements
helped).
The costs are also clear:
- Additional IESG and RFC editor effort in reviewing documents;
- Exhaustion of technical contributors that spend all their time writing
requirements
- Different contributors: writing requirements may tend to attract the
professional standards-goers rather than academics, researchers and
developers.
- Given the average -00 to RFC delay of at least a year, this adds at
least a year to the time-to-completion of the real output, not counting
secondary effects (time spent writing and editing requirements document
isn't available for other tasks).
- Additional opportunities for obstructionism. It is easy to add ever
more requirements or argue, in the abstract, about requirements,
particularly if people are trying to 'position' their prospective solution.
- Delayed failure: Rather than realizing early on that the working group
is hopelessly divided among competing approaches, the requirements
documents papers over this difference until a year+ later or simply
turns it into winning-the-argument-by-exhaustion model of operation.
- Arguments as to whether any actual work follows the letter or the
spirit of the requirement, even though the particular engineering
trade-off may not have been foreseen at the time the requirement was
written.
The last problem is well-known from the waterfall model:
"The disadvantage of waterfall development is that it does not allow for
much reflection or revision. Once an application is in the testing
stage, it is very difficult to go back and change something that was not
well-thought out in the concept stage."
http://searchvb.techtarget.com/sDefinition/0,,sid8_gci519580,00.html
Thus, a pure 'time to first RFC' metric falls short. I wouldn't consider
a WG that spent 24 months producing a requirements document a success,
even if they satisfied some quantitative criteria for timeliness.
To put it less subtly, in some cases we have gone from "rough consensus
and working code" to "fine requirements and forbidden code".
This may still be the right model, but we might want to reflect on this,
rather than making it, without questioning, the model for new efforts.
Henning
More information about the Problem-statement
mailing list