<br><br><div><span class="gmail_quote">On 12/14/06, <b class="gmail_sendername">John C Klensin</b> <<a href="mailto:klensin@jck.com">klensin@jck.com</a>> wrote:</span><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Moving up a few thousand feet, I believe we are missing some<br>things here. In no particular order and with the understanding<br>that I am deliberately not attempting to be precise in order to<br>emphasize the principles...
<br><br>(I got interrupted for several hours before finishing this; one<br>part of it is partially covered by some subsequent discussion,<br>but only part, so please bear with me.)<br><br>(1) As soon as someone says "X is ok because it can be used only
<br>in context Y, even though it would be problematic in other<br>contexts", we have taken ourselves beyond rules about individual<br>code points and into a world in which one must be able to<br>rigorously define Y and the relationship. This would seem to
<br>apply to various separators that would look like prohibited<br>characters in other contexts, such as non-breaking spaces,<br>shape-approximations to apostrophes. It would also apply to<br>any characters that would disappear (become invisible) outside
<br>some specialized contexts in which they have a well-defined and<br>obvious impact on whatever surrounds them, such as ZWJ and ZWNJ.<br>And, finally, this impacts any model that suggests that certain<br>combining characters be permitted only in conjunction with
<br>particular scripts. Look-ahead is not part of IDNA (or<br>nameprep/stringprep) now. While adding it is not infeasible, it<br>isn't a small step.</blockquote><div><br>I agree that it isn't a small step; and we need to be very limited in what we accept. That's why the default-ignorable-code-points are have been added to the list.
<br><br>However, the ZWJ and ZWNJ are required for certain languages in order to represent fairly common words, like the name of a country in the own country's language. I believe that this is worth making an exception for. The complexity can be contained, I believe, by expressing the condition in terms of standard regular expressions, which essentially any implementation has access to. The performance implications are contained, because the number of instances where those regualar expressions need to be invoked will be, as a percentage of all IDNs, extremely low.
<br><br>See <a href="http://www.unicode.org/review/pr-96.html">http://www.unicode.org/review/pr-96.html</a><br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
(2) One of our principles (or, if you prefer, meta-rules) is "no<br>non-language characters". The input from the user/ registrant/<br>good-DNS-behavior side of things has been pretty clear about<br>that. If we have a script that contains a number of
<br>non-language characters, and those characters are not identified<br>by some class or property that permits them to be discriminated<br>from the rest of the script, then we have a problem -- either<br>with the principle or with the way we have so far gone about
<br>this. That is the issue with the IPA Block: clearly it contains<br>some characters we need. Clearly it contains some characters<br>that violate this principle. If we can neither keep it nor<br>eliminate it, either the principle needs to give way or we need
<br>a different criterion or rule set. It is not clear what those<br>might be.</blockquote><div><br>As has been said otherwise, the criteria should generally be based on script, not block. For more on the dangers of script vs block, see
<a href="http://www.unicode.org/reports/tr18/#Character_Blocks">http://www.unicode.org/reports/tr18/#Character_Blocks</a><br><br>Now, as others have said, we could take on the project of going through each of the IPA characters to see which are used in modern languages. There are difficulties with this, as noted. If, however, we really wanted to do this, the much more tractable task would be to (a) *first* identify which we think could cause some significant problem, and only (b) *then* see which of those are used in modern languages. If you want to take a pass at (a), that would be useful.
<br><br>Note that in <a href="http://www.unicode.org/reports/tr39/#References">http://www.unicode.org/reports/tr39/#References</a>, under
<a href="http://www.unicode.org/reports/tr39/data/xidmodifications.txt">xidmodifications.txt</a>, we do have a pass at separating out characters on a character by character basis. However, that file is more directed towards notification of users that there may be a problem (in which case one can be a bit more agressive), than having a hard-and-fast prohibition in the protocol.
<br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">(3) We cannot establish a principle that strings coming into<br>IDNA (or Nameprep) must already be normalized (to NFC at least).
<br>The rule that NFKC(cp) must equal cp is well and good, but,<br>taken by itself, I think it eliminates all sequences involving<br>combining characters for which there are precombined sequences<br>and may have some other ill effects. Am I missing something in
<br>this, or does the rule need further refinement (note that this<br>interacts with (1) above).</blockquote><div><br>There is a common misunderstanding about NFKC and NFC. <br><br>The vast majority of combining marks satisfy the requirement than NFKC(cp) = cp. This requirement does *not* eliminate the requirement to that NFKC(whole_field) = whole_field, which is a *current* requirement of the output of IDNA. If my statement here is too obscure I can elaborate... Have to run to a meeting now.
<br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> john<br><br>_______________________________________________<br>Idna-update mailing list
<br><a href="mailto:Idna-update@alvestrand.no">Idna-update@alvestrand.no</a><br><a href="http://www.alvestrand.no/mailman/listinfo/idna-update">http://www.alvestrand.no/mailman/listinfo/idna-update</a><br></blockquote></div>
<br>