Comments on IDNAbis issues-05

Mark Davis mark.davis at icu-project.org
Thu Jan 10 01:28:27 CET 2008


I sent this almost a month ago, and got no reply. I'm assuming that the lack
of response was due to the holidays, and some discussion or response for
these items will be forthcoming soon.

Mark

On Dec 13, 2007 7:48 PM, Mark Davis <mark.davis at icu-project.org> wrote:

> http://www.ietf.org/internet-drafts/draft-klensin-idnabis-issues-05.txt
> Overview.Many nice improvements to the text.
>
> *This list does not list what was already commented on for * *
> draft-klensin-idnabis-protocol-02.txt*<http://www.ietf.org/internet-drafts/draft-klensin-idnabis-protocol-02.txt>
> * and * *draft-faltstrom-idnabis-tables-03.txt*<http://www.ietf.org/internet-drafts/draft-faltstrom-idnabis-tables-03.txt>
>
> Issues-1. IDNAbis has a major backwards compatibility issue with IDNA2003:
> thousands of characters are excluded that used to be valid. What reason
> might people have to believe that despite the terms NEVER and ALWAYS that
> some future version, IDNAbis-bis, might not also do the same?
>
> Issues-2. IDNA provided for backwards compatibility, by disallowing
> Unassigned characters in registration, but allowing them in lookup. That let
> old clients work despite new software. While once we update to U5.1 that
> is not as much of a problem, it should be made clear why this change is
> made.
>
> Issues-3. In general, whenever a statement is made about some class of
> characters causing a problem, at least one clear example should be provided,
> as in draft-alvestrand-idna-bidi-01.txt<http://www.ietf.org/internet-drafts/draft-alvestrand-idna-bidi-01.txt>
>
> Issues-4. I would strongly suggest separating all of the "why did we do
> this" and "how is it different from IDNA2003" into a separate document. It
> will be of only historical interest after this becomes final, and will then
> only clutter the document.
>
> Details.
> Issues-5.
>
>    IDNA uses the Unicode character repertoire, which avoids the
>    significant delays that would be inherent in waiting for a different
>    and specific character set be defined for IDN purposes, presumably by
>    some other standards developing organization.
>
> Seems odd. There are no other contenders in the wings. Would be better, if
> this has to be said, to just cite other IETF documents describing the
> reasons for using Unicode.
>
> Issues-6.
>
>    To improve clarity, this document introduces three new terms.  A
>    string is "IDNA-valid" if it meets all of the requirements of this
>
>    specification for an IDNA label.  It may be either an "A-label" or a
>    "U-label", and it is expected that specific reference will be made to
>    the form appropriate to any context in which the distinction is
>    important.
> ...
>    A "U-label" is an IDNA-valid string of
>    Unicode-coded characters that is a valid output of performing
>    ToUnicode on an A-label, again regardless of how the label is
>    actually produced.
>
> These definitions appear circular, so they need to be teased out a bit.
>
>
> Issues-7.
>
>    Depending on the system involved, the major difficulty may not lie in
>    the mapping but in accurately identifying the incoming character set
>    and then applying the correct conversion routine.  It may be
>    especially difficult when the character coding system in local use is
>    based on conceptually different assumptions than those used by
>    Unicode about, e.g., how different presentation or combining forms
>
>    are handled.  Those differences may not easily yield unambiguous
>    conversions or interpretations even if each coding system is
>    internally consistent and adequate to represent the local language
>    and script.
>
> I suggest the following rewrite:
>
> The main difficulty typically is that of  accurately identifying the
> incoming character set so as to apply the correct conversion routine.
> Theoretically, conversion could be difficult if the non-Unicode character
> encoding system were based on conceptually different assumptions than those
> used by Unicode about, e.g., how different presentation or combining forms
> are handled. Some examples are the so-called "font-encodings" used on some
> Indian websites. However, in modern software, such character sets are rarely
> used except for specialized display.
>
>
> Issues-8.
>
>    That, in turn, indicates that the script community
>    relevant to that character, reflecting appropriate authorities for
>    all of the known languages that use that script, has agreed that the
>    script and its components are sufficiently well understood.  This
>    subsection discusses characters, rather than scripts, because it is
>    explicitly understood that a script community may decide to include
>
>    some characters of the script and not others.
>
>    Because of this condition, which requires evaluation by individual
>    script communities of the characters suitable for use in IDNs (not
>    just, e.g., the general stability of the scripts in which those
>    characters are embedded) it is not feasible to define the boundary
>    point between this category and the next one by general properties of
>    the characters, such as the Unicode property lists.
>
> There is no justification given for this process. Moreover, it will be
> doomed to failure. Merely the identification of "script communities" is an
> impossible task. Who speaks for the Arabic script world? Saudi Arabia
> (Arabic)? Iran (Persian,...)? Pakistan (Urdu,...)?, China (Uighur,...)?
>
> Issues-9.
>
>       it is removed from Unicode.
>
> (multiple instances)
>
> This is not necessary; characters aren't removed from Unicode. If you
> really have to have it, then add "(however, the Unicode stability policies
> expressly forbid this)"
>
> Issues-10.
>
>    Applications are expected to not treat "ALWAYS" and "MAYBE"
>    differently with regard to name resolution ("lookup").  They may
>    choose to provide warnings to users when labels or fully-qualified
>    names containing characters in the "MAYBE" categories are to be
>
> In practice, expecting applications to treat these differently is wishful
> thinking; especially if it seems Eurocentric to users (see other notes on
> MAYBE). In practice, registries always have the ability to filter characters
> out. See above on removing Maybe.
>
>
> Issues-11.
>
>    5.1.3.  CONTEXTUAL RULE REQUIRED
>
> I know what the point is supposed to be (and don't disagree), but this
> section was very hard to make out.
>
>
> Issues-12.
>
>    Characters that are placed in the "NEVER" category are never removed
>    from it or reclassified.  If a character is classified as "NEVER" in
>    error and the error is sufficiently problematic, the only recourse is
>    to introduce a new code point into Unicode and classify it as "MAYBE"
>    or "ALWAYS" as appropriate.
>
> The odds of this happening are extremely low. Anything in Never has to be
> extremely certain.
>
> Issues-13.
>
>    Instead, we need to have a variety of approaches that, together,
>    constitute multiple lines of defense.
>
> Defense against what? Without examples, it is hard to say what the
> problems are.
>
> Issues-14.
>
>    Applications MAY
>    allow the display and user input of A-labels, but are not encouraged
>    to do so except as an interface for special purposes, possibly for
>    debugging, or to cope with display limitations.  A-labels are opaque
>    and ugly, and, where possible, should thus only be exposed to users
>    who absolutely need them.  Because IDN labels can be rendered either
>    as the A-labels or U-labels, the application may reasonably have an
>    option for the user to select the preferred method of display; if it
>    does, rendering the U-label should normally be the default.
>
> Add:
>
>
>  It is, however, now common practice to display a suspect U-Label (such as
> a mixture of Latin and Cyrillic) as an A-Label.
>
>
> Issues-15.
>
>    6.3.  The Ligature and Digraph Problem
>
>    There are a number of languages written with alphabetic scripts in
>    which single phonemes are written using two characters, termed a
>
>
>    "digraph", for example, the "ph" in "pharmacy" and "telephone".
>
> The text has been improved considerably from earlier versions, but the
> whole issue is just a special case of the fact that words are spelled
> different ways in different languages or language variants. And it has
> really nothing to do with ligatures and diagraphs. The same issue is
> exhibited between theatre.com and theater.com as between a Norwegian URL
> with ae and a Swedish one with a-umlaut.
>
> So if you retain this section, it should be recast as something like
>
>
> 6.3 Linguistic Expectations
>
>  Users often have certain expectations based on their language. A
> Norwegian user might expect a label with the ae-ligature to be treated as
> the same label using the Swedish spelling with a-umlaut. A user in German
> might expect a label with a u-umlaut and the same label with "ae" to resolve
> the same. For that matter, an English user might expect "theater.com" and
> "theatre.com" to resolve the same. [more in that vein].
>
>
> Issues-16.
>
>  there is no evidence that
> they are important enough to Internet operations or
> internationalization to justify large numbers of special cases
> and character-specific handling (additional discussion and
>
> I suggest the following wording instead:
>
>  there is no evidence that
>  they are important enough to Internet operations or
>  internationalization to justify inclusion (additional discussion and
>
> It doesn't actually involve "large numbers of special cases", there are a
> rather small percentage of demonstrable problems in the symbol/punctuation
> area. What we could say is that there is general consensus that removing all
> but letters, digits, numbers, and marks (with some exceptions) causes little
> damage in terms of backwards compatibility, and does remove some problematic
> characters like fraction slash.
>
>
> Issues-17.
>
>    For example, an essential
>    element of the ASCII case-mapping functions is that
>    uppercase(character) must be equal to
>    uppercase(lowercase(character)).
>
> Remove or rephrase. It is a characteristic, but not an essential one. In
> fact, case mappings of strings are lossy; once you lowercase "McGowan", you
> can't recover the original.
>
>
> Issues-18.
>
>    o  Unicode names for letters are fairly intuitive, recognizable to
>       uses of the relevant script, and unambiguous.  Symbol names are
>       more problematic because there may be no general agreement on
>       whether a particular glyph matches a symbol, there are no uniform
>       conventions for naming, variations such as outline, solid, and
>
> Actually, the formal Unicode names are often far from intuitive to users
> of the relevant script. That's because the constraints of using ASCII for
> the name, to line up with ISO standards for character encodings.
>
>
> This section is not really needed. The use of I<heart>NY.com is not really
> problematic; the main justification for removing it is that we don't think
> it is needed (and has not been used much since IDNA was introduced). Better
> to just stick with that.
>
>
> Issues-19
>
>    11.  IANA Considerations
>
>    11.1.  IDNA Permitted Character Registry
>
>    The distinction between "MAYBE" code points and those classified into
>    "ALWAYS" and "NEVER" (see Section 5) requires a registry of
>    characters and scripts and their categories.  IANA is requested to
>
>
> Expecting an IANA registry to maintain this is setting it up for failure.
> If this were to be done, precise and lengthy guidance as to the criteria for
> removing characters (moving to NEVER) would have to be supplied, because of
> the irrevocable nature of this step. The odds of a registry being able to
> perform this correctly are very small.
>
>
> The best alternative would be to simply have all the non-historic scripts
> have the same status in *draft-faltstrom-idnabis-tables-03.txt*<http://www.ietf.org/internet-drafts/draft-faltstrom-idnabis-tables-03.txt>,
> by moving the non-historic scripts to the same status as Latin, Greek, and
> Cyrillic.
>
> The second best would be to have the Unicode consortium make the
> determinations (and take the heat for objections).
>
>
> Issues-20
>
>    Some specific suggestion
>    about identification and handling of confusable characters appear in
>    a Unicode Consortium publication [???]
>
> Use: [UTR36]
>       UTR #36: *Unicode Security Considerations*
>       http://www.unicode.org/reports/tr36/
>
>


-- 
Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.alvestrand.no/pipermail/idna-update/attachments/20080109/e7b43595/attachment-0001.html


More information about the Idna-update mailing list