Response to WG last call, Problem Statement: Thoughts on the IET F problem statement

[Robert Snively]

In consideration of:

	draft-ietf-problem-issue-statement-05.txt

I'm sorry not to have identified this document earlier, but it was brought
to my attention only today by a news publication.

INTRODUCTION:

First let me briefly introduce myself.  I have created and led the
creation of national and international standards in the INCITS, IEEE,
and IETF environments.  I have additionally created defacto standards
in industry interest organizations, both commercial and voluntary.
As a result, I have a fairly diverse background in the creation of
standards and can speak directly to the challenges as seen from
the working group and area levels.  From my experiences, I believe that
I can provide some helpful consideration to the overall IETF process
up to the IESG level.

BASIC ISSUE IS CULTURE:

I believe your IETF draft has correctly identified many problems
and issues.  However, I believe that there is a more fundamental
problem that must be discussed.

Perhaps one of the most important issues is raised in clause 2.1. 

   "The IETF creates standards and is therefore necessarily a Standards
   Development Organization (SDO) but many participants would like to
   differentiate the IETF and its way of working from the 'conventional'
   SDOs which emphasize corporate involvement and mandated delegates."

This attempt to differentiate and create an "academic" culture is perhaps 
the biggest flaw in an organization architecting a world-wide commercial 
infrastructure with very high standards of reliability and security and 
with immense complexity.  From that attempt flow most of the other problems.
The obvious solution, following some of the principles of the other large
successful SDOs, has consistently slipped from IETF's grasp.

PROBLEM STATEMENT: NO MEMBERSHIP REQUIREMENTS

Perhaps the most important word qualifying the internet today is
"commercial".  Companies spend money in their engineering development
organizations to develop and implement equipment in compliance with
IETF documents.  Time to market is a key aspect of such development
efforts.  Complete, accurate, and timely standards are a requirement.
Involving an arbitrary number of standards developers having no commercial
stake in the process of creating a standard is perhaps the largest
single problem in the IETF.

This is touched upon in clause 2.8, but rather than choosing to
guide the culture toward a more effective process, it was treated as
a failure to properly train the participants.  The real issue is that
membership should be earned.

Membership should be earned in each working group or area by attendance, 
by activity, by participating in reviews, and by payment of a fee.  

Membership should be probably be at an organizational level, thus partially
identifying the otherwise hidden interests and agendas of representatives.
The underlying commercial motivation of most participants must be
recognized and brought to light.

PROBLEM STATEMENT: OBSOLETE WORKING RULES

Tradition is a powerful and helpful guide, but must be supplemented with
wisely selected improvements. This is touched on in clause 2.6.5, but its
effects are far-reaching.  It also appears in various guises in clause 2.2.

  OBSOLETE WORKING RULES:  DOCUMENT FORMAT

	It took the ips group over a year to gain
	permission to post drafts in portable document format (pdf).  Even
then
	the documents had to be posted in text as well.  All other standards
	organizations have used sophisticated word processing capabilities
since
	the early 1980s, enabling their documents to contain diagrams
	and relieving the editor of painful document formatting duties.

  OBSOLETE WORKING RULES:  DEFINITIONS OF CONSENSUS

	As you pointed out in clause 2.7, the definition of consensus is
	rather vague.  In every other standards organization there are
formal
	mechanisms that solicit and measure consensus, usually with a
recorded vote
	by organization or member.  That provides a very effective mechanism
	for managing "consensus by exhaustion" issues. 

  OBSOLETE WORKING RULES:  ALL WORK OVER THE INTERNET

	Face-to-face meetings of sufficient duration to do actual work
	provide high bandwidth interchange and understanding among those
	motivated to attend and participate.  As one example, the iSCSI
	activity held two interim face-to-face meetings co-located with the
	INCITS TC T11 SCSI Committee, bringing together the foremost minds
	in SCSI and the Internet.  In each of those all day meetings, more
was
	accomplished in correcting major flaws in the iSCSI document than
	had been accomplished in three months of e-mail interchanges.
	By documenting those meetings in careful minutes, most of the
	decisions made during that period were not challenged later.
	The personal understandings achieved by those meetings were also
	very helpful.

	The IETF meeting weeks, while sometimes helpful, generally do not
have enough
	time allocated to each project within a working group to achieve
helpful technical
	interchange.  At the same time, the number of interested and
knowledgeable 
	participants for each project is a small percentage of those
	attending the working group, making the IETF meetings even less
useful
	for technical interchange.

	You address this briefly in a number of places, but the real issue
	is that the working groups should have autonomy within certain
	limits to hold meetings and teleconferences to accelerate their
work.
	This autonomy issue is partially addressed in several sub-clauses in
2.6.

PROBLEM STATEMENT:  DICTATORIAL STRUCTURE AND TECHNICAL REVIEW

Clause 2.6 and its sub clauses address the following issue, but less
brutally than I would suggest.  Again, this is a flaw in the IETF culture.

In every standards organization I have participated in, except IETF,
the administrative hierarchy is concerned with proper completion
of the technical process, not with the technical content.  The technical
content is worked out by organizations roughly at the level of the
working group in IETF.  As part of the procedure enforced by the
administrative hierarchy, liaison with other working groups, other
Accredited Standards Developers, and with other industry organizations
is established so that architecturally consistent standards are 
developed.  The administrative hierarchy guarantees a fair and open
process and the proper scoping of a standards document.  It also provides
a public review and a public protest mechanism.  It does not 
perform a technical review.

The process of sending documents up through the IESG for technical
consideration shows a lack of trust in the capabilities of the
working group technical participants, a recognition that the process
is unsuccessful in achieving consistent technical development, and
a certain academic arrogance, akin to grading papers.

Without such delegation of the responsibilities for liaison and
technical excellence, IETF will not prove scalable.

This process is also a major impediment to the timely publication
of documents.  As an example, we have about $3 billion dollars worth
of commercial market value backed up against three IETF documents 
that are being held up by a security document that most 
configurations will never use.

PROBLEM STATEMENT:  LACK OF FORMAL RECOGNITION

Clause 2.5.1 suggests that recognition is an important issue.
Formal recognition should be given in every document.  In systems
as complex as the internet, it is absurd to think that there is
one person that authors an RFC.  All meaningful RFCs are the product
of dozens or hundreds of people.  However, the academic background of
IETF, requires (probably because of the "publish or perish" syndrome)
that only one or a few people be listed as authors.  If 
IETF were being honest, it would identify an editor, a chairperson,
perhaps a vice-chair and/or secretary, and cite the full list of 
participating members as authors.  I have heard direct and explicit 
criticisms of this aspect of IETF culture a number of times on several 
IETF documents.

PROBLEM STATEMENT:  NIH

The previous arrogance of IETF administration has prevented it from
looking at very good models for organizational structure and 
standards development, implemented
by such organizations as ANSI and ISO.  It is refreshing that IETF has
finally realized how severe its problems are and is beginning to
cast about for proper solutions.  This was pointed to somewhat more
politely in several sub clauses of 2.6.  This is one part of the culture
that needs to be corrected.

MY RECOMMENDED PROBLEM RESOLUTION, NOT PART OF THE PROBLEM STATEMENT:

My own suggestion would be to
review documents like the INCITS RD-2 (available at www.incits.org),
the ANSI policy documents (available at www.ansi.org), and the ISO
policy documents (www.iso.org) for the many constructive ideas they
have.  Better yet, IETF should recognize that it now representing
a mature industry and become an accredited standards
developer under ISO, IEC, or ANSI, adopting the open, efficient, and
scalable policies of its new organizational parent.

This is not an internet draft, and will not be because of the "obsolete
working rules, document format" issue, mentioned above.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://eikenes.alvestrand.no/pipermail/problem-statement/attachments/20031119/232971ff/attachment.htm