Esszett, Final Sigma, ZWJ and ZWNJ

John C Klensin klensin at jck.com
Mon Feb 23 19:48:57 CET 2009



--On Monday, February 23, 2009 11:05 -0500 Andrew Sullivan
<ajs at shinkuro.com> wrote:

> On Mon, Feb 23, 2009 at 01:06:57PM +0100, Cary Karp wrote:
> 
>> Who said anything about multiple billing?  And, yet again,
>> what makes that a protocol-level concern?
> 
> I agree that making specific bundling recommendations is way
> outside our scope.  I do think, however, that a
> nearly-complete outline of how to solve the problem that
> underpins Paul Hoffman's objection would do an awful lot to
> address that objection.
> 
> With all respect to the Chair, I think that Paul has a
> credible point that the charter can easily be read to restrict
> us from recommending that (possibly a reduced set of) mappings
> end up in the protocol; and also, to restrict bits on the wire
> change (e.g. "Ensure practical stability of validity
> algorithms for IDNs."

Andrew, I hate engaging in protocol-lawyering, especially about
charters and do not believe that the IETF's assumptions about
charters, even with all the use of words like "contract", ought
to block doing something for which there is rough consensus that
it is the right thing by quibbling about charter language.   The
process feels to me like the place where people go to try to
defend their positions after they have lost on substantive
grounds... at least unless the positions are raised very early
in the discussions.  

However, we seem to be there, so...

I'm having trouble understanding your position (and Paul's).
The charter rather specifically says that we can consider and
make incompatible changes:

	"Subject to the more general constraints described
	above, the WG is permitted to consider changes that are
	not strictly backwards-compatible.  For any such change
	that is recommended, it is expected to document the
	reasons for the change, the characters affected, and
	possible transition strategies."

Now, I suppose one could read "- Ensure practical stability of
validity algorithms for IDNs." as one of those "general
constraints" and prohibiting _any_ change that is not strictly
backward-compatible.  But (i) that is explicitly an "additional
goal", not a "general constraint".   And (ii) even if it were a
"general constraint", reading it to prohibit this case would, I
believe, require reading it to prohibit _any_ change that is not
strictly backward-compatible with IDNA2003 and that would
completely contradict the provision quoted above.

That leaves the argument that this is invalid under the charter
depending on one of

(i) the claim that permitting Eszett disturbs "the current use
and operation of the domain name system"... But no IDNA change
can do that, since whole premise of IDNA is that it is invisible
to DNS operations and the actual DNS continues to see only
hostname-style labels,

(ii) the claim that eliminating the mapping of these character
(which the charter requires) violates the provision that "the
DNS [to] continue to allow any system to resolve any domain name
in a consistent way"... But, again, this is about the domain
names in the DNS, as evidence by the use of "any system".   But
an ACE-based string that is valid under IDNA2003 will resolve to
exactly the same thing and in exactly the same way in IDNA2008.
An ACE-based string that contains coding for Eszett or Final
Sigma is not valid under IDNA2003 and would presumably fail
ToASCII, but IDNA2003's "resolve everything" rules would,
curiously, make it compatible (and resolve identically) with
both versions of the protocol _as far as resolution is
concerned_.   Moreover, if one cannot _add_ Eszett and Final
Sigma as characters that are represented as themselves, then I
don't see how one can add any character that is not supported by
IDNA2003 without violating that reading of the statement in the
protocol, leading to a silly state vis-a-vis this WG being
chartered at all (or chartered to do anything other than fix
Bidi and tune up definitions... and the charter is certainly not
that limited).

(iii) that there is an implicit "on-the-wire" provision in the
charter that prohibits any change to any character that is
mapped to another one by IDNA2003.   But there is nothing in the
charter that makes Stringprep mappings derived from CaseFold any
different from those derived from Compatibility character
considerations. So that interpretation would essentially
prohibit getting rid of mappings, which the charter requires.
In addition, anything on the DNS wire is an A-label, turning
this argument into a special case of (ii), so there is just no
charter issue.

>  I will note, however, that I always had
> the impression that the Eszett and Final Form Sigma changes
> were under consideration, and I'm not personally really
> conviced by the "restrictive reading").

Certainly they have been since long before the WG came into
being, even though the charter did not include a specific list
of characters being considered.

>  If that reading is
> correct, then I don't think it matters whether the WG happens
> to agree that the change is safe: it's still outside of
> charter, so the WG is not allowed to adopt such a
> recommendation without warning the rest of the IETF that's
> what we're going to do.  The way to make such a warning is by
> charter change.

> So, assuming we're not going to recharter, I think a detailed
> explanation of how the change can be managed by registries
> would be extremely helpful.  That is because, given a
> restrictive reading of the charter, some people who are not
> following this WG may be very surprised to learn that such an
> incompatible change is getting included.  Those people might
> object very late in the process -- at IETF last call or
> (worse) later.  I don't know about others, but my experience
> is that the later this sort of big objection comes, the worse
> it is for the documents.  I think it'd be nice to have
> something in hand to answer such objectors, if they show up.
> But I'm not, note, volunteering to do the work (and I
> appreciate that that's a good reason others might ignore this
> suggestion).

Rationale already discusses options for registries to manage
this change, as required by the provision for
non-backward-compatible changes quoted above.  Moving beyond
identifying that list of possible options to making an explicit
recommendation /requirement to registries is, IMO, not only
outside the scope of the charter but that of the IETF.  Where
Rationale may not go far enough is in explicitly saying what has
been said on this list before: that this is being changed
because applying CaseFold in cases where it doesn't produce a
simple upper->lower case mapping creates confusion, violates a
strong recommendation from TUS, because removing mappings
produces the inclusion of those characters as a consequence, and
because the most relevant registries want the flexibility that
having the distinct character available provides (in the Greek
case, assuming that they can't preserve the  mapping).  If
people want that added, I'm happy to do so, ideally using text
suggested by others.

But, again, I just don't understand the "charter violation"
argument if one reads the whole charter rather that taking a few
sentences out of context and interpreting them independent of
all of the rest.   If you still do, or think you understand that
interpretation, please explain it further.

     john



More information about the Idna-update mailing list