Prohibiting mapping of PVALID characters

[Paul Hoffman]

At 4:18 PM -0500 12/9/09, John C Klensin wrote:
>Since I'm trying to look at text this afternoon and evening, is
>there agreement that it would be wise to add a "SHOULD NOT
>change the interpretation of valid strings by mapping PVALID
>characters to anything else" to the portion of Protocol where
>mapping is mentioned, or should (sic) I leave well enough alone?
>If the former, is that a correct statement of the proposition?
>
>Note that such a statement would make Sharp-S to "ss" mapping
>not-fully-conformant to IDNA2008, even though "SHOULD" does
>leave an opening...

draft-ietf-idnabis-protocol-17.txt mentions mapping in only two places.

4.1, first paragraph:

   . . . Entities responsible for
   zone files ("registries") MUST accept only the exact string for which
   registration is requested, free of any mappings or local adjustments.

5.2:

   The string is converted from the local character set into Unicode, if
   it is not already in Unicode.  Depending on local needs, this
   conversion may involve mapping some characters into other characters
   as well as coding conversions.  Those issues are discussed in
   [IDNA2008-Mapping] and the mapping-related sections (Sections 4.4, 6,
   and 7.3) of [IDNA2008-Rationale].  The result MUST be a Unicode
   string in NFC form.

The words in 4.1 are quite definitive, irrespective of whether or not the string contains PVALID characters. If you want to add something, it would be in 5.2. A proposed change to the second sentence would be:

Depending on local needs, this conversion may involve coding
conversions. The conversion may also involve mapping some characters;
however, such mapping MUST NOT be applied to any character that has a
type of PVALID, regardless of the local needs.

The problem with this is that it makes a MUST-level requirement that is begging to be broken ("our needs for mapping are greater than our needs for conformance to the RFC"). But it might be the best we can do.