Latency

[Henning Schulzrinne]

Among many other limitations of the -00-to-RFC data is a more 
macro-scale one that I don't think has been discussed: in many working 
groups that are developing new protocols, the development model has 
changed. (This does not tend to affect our staple working groups that 
are effectively in protocol maintenance mode.)

If we take the software engineering analogy, the IETF has basically gone 
from 'rapid prototyping' to the classical waterfall model 
(http://c2.com/cgi/wiki?WaterFall). Almost any working group has to now 
go through a much longer chartering process, fairly extensive 
'requirements' and 'framework' documents, before the working group is 
even allowed to talk about protocol development. This is a process 
change which was (I believe) never really discussed and isn't really 
documented.

Thus, I think the perception of long latency is not merely an issue of 
publication delay but also that the whole lifecycle takes longer since 
it is now composed of multiple such publication cycles.

There may well be reasons for this approach, but it might be helpful to 
weigh its costs and benefits. The benefit is presumably better output - 
the resulting work looks much different (and better) than it would have 
without this effort. It would be nice to have concrete examples for that 
(e.g., by asking the authors of the final spec how much the requirements 
helped).

The costs are also clear:

- Additional IESG and RFC editor effort in reviewing documents;

- Exhaustion of technical contributors that spend all their time writing 
requirements

- Different contributors: writing requirements may tend to attract the 
professional standards-goers rather than academics, researchers and 
developers.

- Given the average -00 to RFC delay of at least a year, this adds at 
least a year to the time-to-completion of the real output, not counting 
secondary effects (time spent writing and editing requirements document 
isn't available for other tasks).

- Additional opportunities for obstructionism. It is easy to add ever 
more requirements or argue, in the abstract, about requirements, 
particularly if people are trying to 'position' their prospective solution.

- Delayed failure: Rather than realizing early on that the working group 
is hopelessly divided among competing approaches, the requirements 
documents papers over this difference until a year+ later or simply 
turns it into winning-the-argument-by-exhaustion model of operation.

- Arguments as to whether any actual work follows the letter or the 
spirit of the requirement, even though the particular engineering 
trade-off may not have been foreseen at the time the requirement was 
written.

The last problem is well-known from the waterfall model:

"The disadvantage of waterfall development is that it does not allow for 
much reflection or revision. Once an application is in the testing 
stage, it is very difficult to go back and change something that was not 
well-thought out in the concept stage." 
http://searchvb.techtarget.com/sDefinition/0,,sid8_gci519580,00.html

Thus, a pure 'time to first RFC' metric falls short. I wouldn't consider 
a WG that spent 24 months producing a requirements document a success, 
even if they satisfied some quantitative criteria for timeliness.

To put it less subtly, in some cases we have gone from "rough consensus 
and working code" to "fine requirements and forbidden code".

This may still be the right model, but we might want to reflect on this, 
rather than making it, without questioning, the model for new efforts.

Henning