DRAFT Status of Work on IDNA2008 + IDNAv2

YungJin Suh yjsuh at nida.or.kr
Wed Mar 25 10:59:40 CET 2009


Dear all WG members,
 
About the JAMO characters in Korean, We still strongly recommend to disallow
these characters.
 
We want to allow ONLY Hangul Syllables(U+AC00 ~ U+D7A3) in revised IDNA.
 
I attached the letter from Korean goverment. (I think some of you may had
already read it .)
 
I hope this document helps you to understand our situation.
 
With regard to the local mapping draft,
 
About 'dots' as label separators, actually Korean doesn't have mapping
problems. But Chinese and Japanese do. 
So I hope this problem will be solved in some way or other.
 
And about 'Compatibility characters', we defined Korean IDN in the draft as
following:
      The term "Korean IDN" stands for "IDN consists from CJK scripts marked
with 'Y' in 'K' column, which is Hangul Syllables(U+AC00 ~ U+D7A3),  and
LDH".   Permitted characters in Korean IDN are listed in
[IANA-IDN-Language-ko-KR].
Eventhough in IDNA2003 allowed JAMO in Korean, we defined like this. 
Because if we restrict the range of allowed characters to only Hangul
Syllables(U+AC00 ~ U+D7A3), nomalization is not an issue for Korean IDN
anymore.
 
Again, allowing only  Hangul Syllables in IDNA is recommended.
 
Thank you.
 
With regards,
 
YungJin Suh
=======================
YungJin Suh
Head of DNS section, KRNIC, NIDA
yjsuh at nida.kr <blocked::mailto:yjsuh at nida.kr> 
+82-2-2186-4562(O)
+82-10-4820-8291(M)
========================
 

  _____  

From: idna-update-bounces at alvestrand.no
[mailto:idna-update-bounces at alvestrand.no] On Behalf Of Vint Cerf
Sent: Monday, March 23, 2009 5:31 AM
To: Mark Davis
Cc: idna-update at alvestrand.no
Subject: Re: DRAFT Status of Work on IDNA2008 + IDNAv2


thanks Mark - I re-issued version 5 of the material so some of your comments
have crossed in the mail. I will try to update once more to take into
account your comments or at least annotate as reminders for discussion. 

v


Vint Cerf
Google
1818 Library Street, Suite 400
Reston, VA 20190
202-370-5637
vint at google.com




On Mar 22, 2009, at 3:51 PM, Mark Davis wrote:


Here are comments on the status. (I tried to update to the later doc, but
because it was only distributed in pdf, I had to do it manually, so I may
have missed something.) 


Mark



On Fri, Mar 20, 2009 at 06:12, Vint Cerf <vint at google.com> wrote:


DRAFT Status of work on IDNA2008

3/21/2009 0523 PDT


Vint Cerf


This brief summary is intended to provide some focus for the IDNABIS WG
meetings
scheduled for Monday and Tuesday, March 23 (1740-1940) and March 24
(0900-1130).

One goal is to try to assess rough consensus about the present documentation
on the
presumption that we are abiding by the ground-rules set forth in the charter
of the WG.
Another is to assess what the implications are for users, registries,
registrars if
IDNA2008 is adopted as it presently stands.  A third goal is to examine the
implications
of the IDNAV2 proposal from Paul Hoffman and contrast with adoption of
IDNA2008.

I fully recognize that consensus has to be assessed from mailing list
exchanges, not
merely from appearances at our face to face meetings.

The material presented below is by no means intended to be more than a basis
for
discussion, and is not intended as a penultimate recommendation.


I think it would also be useful to mark where each is different from
IDNA2003. 



Background





Under the IDNABIS charter, the IDNA2008 design as it now stands makes
several
specific assumptions or makes specific propositions to achieve a number of
goals:

0. Avoid dependence on any specific version of Unicode through the use of
rules
   for determining PVALID characters based on Unicode character properties

 
add: "as much as possible". Exceptions may be necessary in some cases (and
are included in the draft tables).



1. No change to the deployed DNS server functionality (domain name labels
limited to
   ASCII and case-insensitive matching only) 

[no change from IDNA2003] 


2. Esszet, Final Sigma, ZWJ and ZWNJ, geresh and gershayim are PVALID
characters
    some of which are treated through contextual rules (there is still
ongoing discussion
    about the implications of these choices)


This is also a current feature of the drafts, but not required by the
charter. It is unclear whether this is actually consistent with the charter
or not. "This work is intended to specify an improved means to produce and
use stable and unambiguous IDN identifiers." Effectively, any IDN with the
first four characters is ambiguous between versions of IDNA in that it will
lead to different addresses. 




3. Unassigned Unicode characters will not be looked up

 
Just a comment (no change): IDNA2003 had the slightly different goal:
unassigned Unicode characters will not be returned from the DNS. 



4. No mapping of characters at least within the protocol specification 


5. No modification of or dependence on Nameprep  (and thus no impact
  on other protocols relying on Nameprep or Stringprep.) 


6. Clear specification of valid "dot" form in a way that is consistent with
DNS
    protocol requirements.


IDNA2003 specified the dot form in a way that is consistent with DNS; that
is, it required no change of the DNS protocol, so this is no change. That
is, once in the ACE form, dots are dots.



7. Symmetry between native-character ("Unicode") and ACE ("Punycode")
    forms of a label.


This may be a goal, but it is not achieved by the current drafts. There is a
strong asymmetry between them in that in lookup, an implementation need not
check that what appears to be an A-Label is one, but it must check that a
U-Labels is one (mostly). (Comment: I believe that this should be a goal: if
it is important to check those requirements, then it is important to test
both A and U Labels; if it is not important to test them, then it should not
be a requirement for either one.)




8. Conversion to an inclusion list of PVALID characters (as distinct from
the
   IDNA2003 posture that excluded only a few Unicode characters) 


9. Improved terminology to make categories and types of labels more clear.
   (Definitions) 


10. Provide explanation for decisions and their motivations (Rationale) to
    aid implementors, registries, registrants and users in understanding
IDNA.

 
Rationale doesn't really provide explanation for motivations in enough
detail to be useful. I'd recast this as: "Provide informative background
material (Rationale) to aid ..."



11. Separately describe registration and lookup procedures to improve
clarity

 
The goal is good, but the current drafts don't meet the goal. Whether it
increases clarity or not is unclear, since by doing so makes it difficult to
determine what the similarities and differences are between the two
processes. So drop "to improve clarity". (A relatively small recasting of
the text to make it precisely parallel between them (including numbering),
and point out precisely where the differences are, would meet this goal.




12. Specify tests to be applied at lookup time in an attempt to limit abuse
of
      IDNA at all levels of registration


That is not a change from IDNA2003. The tests are different, and are
expanded, but it is a quantitative difference, not qualitative. For example,
IDNA2003 did test bidi; we just think the IDNA2008 tests are better. And the
"in an attempt to limit abuse" is not true; the changes in IDNA2008 will
have a trifling effect on abuse at the very best, and introduce significant
opportunities for spoofing because of the 4 ambiguous characters. And
affecting the "phishing" problem is not a requirement of the charter. So
this item should be removed.
 



13. Clarify what is expected of IDNA-aware applications and domain name
      "slots" with regard to invalid labels and future extensibility


These are still not nailed down in the current drafts. My expectations are
that once a domain name is valid, it remain valid for all time -- that is,
we are doing a one-time massive compatibility change, but there will be no
more changes that would affect compatibility. However, that is not captured
in the text, despite the charter requirement "This work is intended to
specify an improved means to produce and use stable and unambiguous IDN
identifiers."

Another major change is the introduction of a mechanism for changing IDNAs
on the fly via the context mechanism, with and associated process.






Chartering and Re-Chartering

(1) A Re-charter is needed if we abandon a significant fraction of the
IDNA2008 goals
and methods. IDNAv2, as described by Paul Hoffman requires a re-charter.

(2) A Re-charter is needed if the WG decides to introduce mappings into the
IDNA2008
specifications since the basic assumption in IDNA2008 was that mapping would
not
be part of the specification.

(3) It is possible that re-charter might not be needed if IDNA2008 adopts
some
IDNA2003 operations under a restricted set of conditions and only at lookup
time for purposes of easing the transition to IDNA2008. This would be up to
the
AD and IESG presumably to decide.

Basics for IDNA2003 and IDNA2008

Both of these specifications use the Punycode algorithm to generate what
IDNA2008 would call an A-label (ie. "xn-- <LDH compliant string>") from


Better expressed as an XN label. That terminology can be applied to both,
while A-Label only makes sense for IDNA2008. 


labels expressed as a string of characters drawn from a subset of Unicode
defined characters.

DNS matching is done in the servers by comparing the query string to the
registered string in a case-independent fashion.  For IDNs, these
comparisons
are done after conversion into the "xn--" prefix form. For IDNs the case
insensitive
matching of the DNS servers applies only to the A-label form and not to the
Unicode form. This means that the case-insensitive matching behavior of
in traditional ASCII labels is not conferred on IDNs in their Unicode form.

The case-insensitive comparisons between traditional LDH domain names is
approximated under IDNA2003 by using CaseFold as a mapping guide on the
Unicode strings being looked up. In addition, IDNA2003 also maps the
so-called
"compatibility" characters of Unicode into their counterparts. The same
actions

 
=> "compatibility decomposable" characters of Unicode into their
counterparts
[Not all compatibility characters are decomposable and vice versa.]
 


precede the registration of new domain names under IDNA2003.

Unicode CaseFold maps to upper case and then map back to lower case.

 
This is not quite accurate; better would be to say "Unicode CaseFold maps
characters to lowercase values based on an an equivalence class formed by
including lowercase, uppercase, and titlecase mappings."



Prior to Unicode 5.0, Ezsett became "SS" because there was no upper
case, then became "ss" in the lower case mapping.  Under Unicode 5.0
CaseFold was unchanged for  stability reasons. Consequently
CaseFold (ESSZETT) is "ss" rather than lower case esszett even after
the introduction of upper case ESSZETT in Unicode 5.0.


=>
The uppercase of ezsett in Unicode is "SS", following national standards and
practices. As of Unicode 5.1, an uppercase version of eszett became
available. Under the Unicode case folding, both map to "ss".




Under IDNA2003, both DISALLOWED and UNASSIGNED characters
are looked up. If abusive registrations are made using DISALLOWED
or UNASSIGNED characters, these registered domain names may be
be found on lookup by IDNA2003-compliant clients.


This is not correct, as Erik points out.
 



Under IDNA2008, UNASSIGNED and DISALLOWED characters are not looked up.
If new characters become defined under a new version of Unicode
an old client will not look them up until it is updated. Abusive
registrations
using UNASSIGNED characters will not be looked up.

Script mixing is not banned under IDNA2003. Under IDNA2008, BiDi
bans mixing of European and Extended Arabic-Indic numbers with
Arabic numbers.  That is AN and EN characters may not be present in
the same label. Otherwise, mixing is permitted in IDNA2008.

IMPLICATIONS OF ADOPTING IDNA2008 AS CURRENTLY SPECIFIED


1. IDNA2008 is case sensitive for labels with non-LDH characters in them but
is

... with at least one non-LDH character... 


case-insensitive for LDH characters

for example" buecher "is all ASCII and could be matched with "Buecher" or
"bUecher"
under IDNA2008

however "B<u-umlaut>cher" would not be allowed because Tables (see 4.2.2)
would
disallow Latin Capital letters. Some users accustomed to LDH-label behavior
may be surprised that "B<u-umlaut>cher" and "b<u-umlaut>cher" do not match.

On the other hand, the symmetric relationship between the IDNA2008-defined
A-Label and U-Label has the benefit one can use exact match for either
U-label form or A-label forms since they are directly and unambiguously
transformable into each other.

 

However, this symmetry will not exist for

cases where the IDNA2003 A-Label and IDNA2008 A-label for the same

U-Label differ. [Query: will this be a material problem only for actual

registrations under IDNA2003 that differ in A-label form from IDNA2008?]


For registries, this is an advantage (equivalent to disallowing mapping),
but it is not so clear that it is a "benefit" for lookup.





2. IDNA2008 does not ban script mixing even within labels.

Attempts to fashion rules along these lines have run into problems
in which characters that may be confused for others are needed
to express strings in particular languages. The International Phonetic
Alphabet (IPA) characters are a case in point. Some are used for
certain (e.g. African) languages but some of these characters
can be confused for others in the Latin alphabet. Other examples
exist in Arabic, Cyrillic, Greek among others.

Even in the absence of intra-label script mixing, inter-script confusion
such as the Russian word for "restaurant" looking like  "pectopah" in
Latin characters is quite possible.

Despite the apparent desirability of such a ban at protocol level, there
are simply too many combinations of confusion within-scripts and between
scripts to benefit significantly from a protocol-level ban. On the other
hand,
registry level constraints that may be more script-aware appear to be
the most effective tool we have.


I think client-level warnings are the most effective constraint. After all,
if we could always trust the registries, we would need *no* constraints on
the client side in the protocol.
 



3. Esszet is permitted and its usage appears to be geographically and
language
specific. Under IDNA2003, this character is mapped into "ss". To deal with
the
potential conflict with previously mapped registrations in which Esszet is
mapped
to "ss" registries would need to appeal to Rationale 7.2 options, for
example,
to deal with this. Note that not all collisions may be a consequence of
mapping, i.e.,
many occurrences of "ss" in German text are not typographic variations of
Esszett and very few occurrences in Latin script, without consideration of
language,
are variations of Esszett either.

4. Final Sigma is permitted and raises similar issues to Esszet with regard
to
collisions and the same remedies would apply.

5. ZWJ/ZWNJ

In IDNA2003, these characters were mapped to "nothing". It has become
apparent
however that some Indic scripts need them. Persian registries currently
reject registration of labels including ZWJ/ZWNJ although ZWNJ is used in
writing Persian languages. Arabic language does not need ZWJ/ZWNJ.
Mapping to "nothing" in INDA2003 has the side-
effect of inhibiting domain name expression in some Indic scripts including
Tamil and Devanagari. Permitting either or both as valid characters creates
a compatibility problem similar to the Esszett one; i.e., one cannot tell
whether a DNS label, when converted back to native character form, was
intended to be written with ZWJ, ZWNJ or neither.

Elaboration: Suppose that "ab" is a string in one of the scripts in which we
now
propose to permit ZWNJ.  All we have in the DNS is the A-label equivalent of
"ab".
We can't tell from looking at it whether the starting string, as
seen/preferred by the
registrant, was
 ab    or
 aZWJb
since both map to the same A-label.

Under IDNA2008, if the user enters "ab", she gets one A-label
while, if she enters "aXWJb", she gets a different A-label.
That is exactly the same as the Eszett problem -- you can't tell
from the IDNA2003 A-label what the original intention was and
use of the string under IDNA2008 gets you a different A-label
than it does under IDNA2003.

Joiner characters become invisible if inserted in strings written in scripts
that do not use them. 

=> in strings where they make no visual difference. This included scripts
that do not use them, and many positions in scripts that do use them. 

Unicode classifies these characters
as "COMMON" so they also end up passing any plausible tests to prevent
mixing of scripts in a label. Contextual rules are needed to restrict their
use
to strings in scripts where they have some effect. 


where they could have some effect (they won't always, and even when they
commonly have an effect, it depends on the font).
 

We end up relying on
registries to adopt their use judiciously within those scripts. See also the
Rationale document for further commentary.

6. Symbols and punctuation are NOT PVALID under IDNA2008 but are valid

Most symbols... 


under IDNA2003 leading to a variety of potential confusions with
"slash-like"
symbols or other symbols used in URIs for example. IDNA2008 rules reduce
confusion potential by making all characters with these Unicode properties
invalid for use with Domain labels.

either most, or add at the end "with certain exceptions" 



It is not clear that such symbols are critically needed for domain names.

Another reason for banning these characters is that they complicate
references, discussions and databases (such as WHOIS) because it is
not clear how to describe them in common, informal usage.
What is the correct way to refer to "-" ? Is it "hyphen", "minus sign",
"hyphen-
minus" or "short middle horizontal bar?" And is "." "period", "dot", "full
stop",
or something else? What about "#" - is it "pound", "hash", "number sign" or
"tic-tac-toe"? "Heart" is another example: which one is it?


Thus is just not an issue; there are thousands of letters that have
ambiguous or multiple names. This paragraph just can't be fixed; it needs
removal.
 



To be fair, one could refer to the Unicode long name for the character or
even
the "U+" form although this sounds pretty awkward in practical terms.

7. JAMO characters in Korean have been made Protocol Invalid (DISALLOWED)
for reasons similar to (6) above. They introduce a combinatorial explosion
of different
string representations built from JAMO primitive characters. They are valid
under IDNA2003.


This is debatable. The only reasonable rationale we can include is that they
are only used in historic Korean. 



8. Under INDA 2008, when a new version of Unicode is released the following
steps can be taken:

a. review of changes that might require new rules in the IDNA2008 framework.
Such a conclusion would assuredly require formation of a  WG to facilitate
\new RFC
production. This is thought to be extremely unlikely to happen.

b. A review of changes might only require exception rules to preserve
compatibility. It is possible that the required changes might be delegated
to an IANA action possibly in consultation with an expert committee
to generate new tables.


The current drafts require new RFCs in order to change the exception tables,
I believe. It would be better to change that to have the exception table
governed by the same process as the context tables (under stability
provisions).




c. Generate new tables for IANA registry (suitable for downloading as needed

During the transition some will clients have the older tables and
some registries the newer ones. Lookups of Domain Names containing
new PVALID characters by older clients will fail under IDN2008 because
the client will reject UNASSIGNED characters until the clients are updated
with the new PVALID characters.


That is not the bad part of the transition. The bad part is that old
characters may transform from DISALLOWED to PVALID only during the
transition, then corrected, or transform from PVALID to DISALLOWED only
during the transition, then corrected. And the correction period may be
long, depending on when software is updated. That is, if a program ships
every two years, and is updated during the correction, it will be wrong for
2 years.



... 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.alvestrand.no/pipermail/idna-update/attachments/20090325/1d3d2d2e/attachment-0001.htm 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: Comments for Korean IDN.pdf
Type: application/pdf
Size: 196270 bytes
Desc: not available
Url : http://www.alvestrand.no/pipermail/idna-update/attachments/20090325/1d3d2d2e/attachment-0001.pdf 


More information about the Idna-update mailing list