<font size="2"><br clear="all">Mark<br><br><br></font>
<div class="gmail_quote"><font size="2">On Mon, Jun 29, 2009 at 13:26, Vint Cerf <span dir="ltr"><<a href="mailto:vint@google.com" target="_blank">vint@google.com</a>></span> wrote:<br></font><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div><font size="2">Mark, et al,</font><div><font size="2"><br></font></div><div><font size="2">broad re-mapping, even if only for lookup, makes me wonder about ways to mislead users through the use of labels in domain names that would not be allowed until mapped. I think I can understand how we might want NOT to map some of the recently introduced characters (sharp-s for instance) and how we might want to map upper case into lower case prior to lookup (emulating the case independent matching of the purely ASCII domain names/labels of the past. I am having some difficulty with the full range of characters that might be invalid under IDNA2008 but mapped into valid IDNA2008 characters. If there has been a trend in the discussions it has been towards limiting the set of characters that may be mapped prior to lookup. </font></div>
</div></blockquote><div><font size="2"><br>This does limit those characters, does limit the characters remapped, and <i>is</i> a compromise (as I said, I'd rather just map them all; it would be the simplest and offer the most compatibility).<br>
<br>Let me provide some figures to illustrate this.<br><br>Caveats: sizes may change depending on tweaks/changes to the formulation, etc.<br><br></font>
<table style="border-color: rgb(136, 136, 136); border-width: 1px; border-collapse: collapse;" border="1" cellspacing="0"><tbody><tr>
<td style="text-align: left;"><font size="2"><b>Size<br>
</b></font>
</td>
<td><font size="2"><b>Description<br>
</b></font>
</td>
<td style="text-align: center;"><font size="2"><b>Clause</b><br></font>
</td>
</tr>
<tr>
<td style="text-align: right;"><font size="2">90,261</font></td>
<td><font size="2">PVALID or CONTEXTx</font></td>
<td style="text-align: center;"><font size="2"> </font></td>
</tr>
<tr>
<td style="text-align: right;"><font size="2"> 1,023,851</font></td>
<td><font size="2">Not PValid or ContextO</font></td>
<td style="text-align: center;"><font size="2">1<br></font>
</td>
</tr>
<tr>
<td style="text-align: right;"><font size="2"> 5,815</font></td>
<td><font size="2">& NFKC/Case mapped<br></font>
</td>
<td style="text-align: center;"><font size="2">2<br></font>
</td>
</tr>
<tr>
<td style="text-align: right;"><font size="2">4,513</font></td>
<td><font size="2">& LetterDigit or Pd<br></font>
</td>
<td style="text-align: center;"><font size="2">3<br></font>
</td>
</tr>
<tr>
<td style="text-align: right;"><font size="2">4,312<br></font>
</td>
<td><font size="2">& Results are PValid or ContextO<br></font>
</td>
<td style="text-align: center;"><font size="2">4<br></font>
</td>
</tr>
<tr>
<td style="text-align: right;"><font size="2">2,324<br></font>
</td>
<td><font size="2">& Results are allowed DTs & not Hangul<br></font>
</td>
<td style="text-align: center;"><font size="2">5,6</font></td></tr></tbody></table><font size="2"><br>I reordered the proposed criteria for REMAP to make the above clearer:</font><br>
<ol><li>The character is not PVALID or CONTEXTO</li><li>The character is mapped by NFKC_CaseFold</li><li>The character is a LetterDigit or Pd<br>
</li><li>If remapped by the Unicode property NFKC_Casefold*, then the resulting character(s) are all PVALID or CONTEXTO</li><li>The character has one of the following Decomposition_Type values:
canonical, initial, medial, final, isolated, wide, narrow, or compat</li><li>The character does not have the Script value: Hangul</li></ol>I also put a breakdown of that the mapped characters on <a href="http://www.macchiato.com/unicode/idna/remap" rel="nofollow">http://www.macchiato.com/unicode/idna/remap</a>, and updated the text there for John's comments.<br>
<font size="2"><br></font></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div><div><font size="2"><br></font></div><div><font size="2">I think we need to find a space around which compromise and consensus can built as to what chars are allowed to be mapped prior to look up. I think we all agree that there should be no implicit or explicit mapping in the registration process. </font></div>
</div></blockquote><div><br>Yes, agreed; that was just an oversight on my part.<br> <br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div><div><font size="2"></font></div>
<div><font size="2"><br></font></div><div><font size="2">Looking for more common ground.</font></div></div></blockquote><div><br>Me too.<br> <br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div><div><font size="2"></font></div><div><font size="2"><br></font></div><font color="#888888" size="2"><div>vint</div></font><div><div><div><font size="2"><br></font></div><div><font size="2"><br></font><div><div><font size="2">On Jun 29, 2009, at 3:46 PM, Mark Davis ⌛ wrote:</font></div>
<font size="2"><br></font><blockquote type="cite"><font size="2"><br clear="all">Mark<br><br><br></font> <div class="gmail_quote"><font size="2">On Mon, Jun 29, 2009 at 10:55, John C Klensin <span dir="ltr"><<a href="mailto:klensin@jck.com" target="_blank">klensin@jck.com</a>></span> wrote:<br>
</font>
<blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><font size="2"> Mark,<br><br> Several comments inline...<br><br> --On Sunday, June 28, 2009 21:28 -0700 Mark Davis ⌛<br>
</font>
<div><font size="2"><<a href="mailto:mark@macchiato.com" target="_blank">mark@macchiato.com</a>> wrote:<br><br> > Returning to the discussion, now that some of my other<br> > standards work is under control (RFC4646bis was approved,<br>
> whew!)<br></font> </div><font size="2">>...<br></font> <div><font size="2"><br> > Now, my position is still that the simplest and most<br> > compatible option open to us is to simply map with NFKC +<br> > Casefold.<br>
<br></font> </div><font size="2">I continue to believe that CaseFold is a showstopper. When its<br>
results are not identical to those produced by LowerCase, it<br> produces results that are astonishing to some users and leads us<br> into the "is that a separate character or not" trap that we've<br> seen manifested at least twice. I note that TUS recommends<br>
against its use for mapping (as distinct from comparison) and<br> appears to do so for just the reason that it involves too much<br> information loss.</font></blockquote><div><font size="2"><br>You need to provide actual data behind this. Please list exactly the characters that you mean, and why you think they are problematic. Note also that the formulation that I gave means that any character that is PVALID would automatically be excluded, eg if final-sigma is PVALID then it is unaffected. And we can certainly introduce other exceptions.<br>
<br>And I know full well about the issues in TUS, having written or participated in the writing of them.<br><br></font>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><font size="2">
>...<br></font> <div><font size="2">> Proposal: A. Tables document<br> ><br> > Add a new type of character: REMAP. A character is REMAP if it<br> > meets *all of * the following criteria:<br> ><br></font> </div>
<font size="2">> 1. The character is not PVALID or CONTEXTO<br>
> 2. If remapped by the Unicode property NFKC_Casefold*, then<br></font> <div><font size="2">> the resulting character(s) are all PVALID or CONTEXTO<br></font> </div><font size="2">> 3. The character is a LetterDigit or Pd<br>
> 4. The character has one of the following<br></font>
<div><font size="2">> Decomposition_Type values: initial, medial, final,<br> > isolated, wide, narrow, or compat<br><br></font> </div><font size="2">I am very concerned that collapsing initial, medial, and final<br>
together may get us into problems with other language<br>
communities similar to those we have gotten into with Final<br> Sigma, especially when those communities denote word boundaries<br> by the appearance of final or initial forms and hence would use<br> those forms in a style similar to the way "BigCompany" or<br>
"big-company" might be used in ASCII.</font></blockquote><div><font size="2"><br>The mechanism used to indicate boundaries is not, as you think, the use of the presentation forms; it is the use of the ZWNJ/J, which we already provide for.<br>
<br></font>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><font size="2"><br><br> As I've said several times before, even if we disallow the<br>
NFKC-affected forms those characters, if a need arises, we can<br>
(painfully) redefine them as PVALID and allow them. But, if we<br> map them to something else, we lose all information about what<br> was intended/desired and end up in precisely the mess we have<br> with e.g., Final Sigma and ZWJ/ZWNJ in which "the right thing<br>
to do" poses enough compatibility problems to cause opposition<br> to making changes.</font> </blockquote><div><font size="2"><br>You make it sounds like final sigma, ZWJ/NJ, eszett and the other cases under discussion were oversights in the process of developing the current IDNA. That wasn't the case; these were deliberate choices made at the time. A case mapping is also a 'loss of information', but one that people clearly want.<br>
<br>If you have any particular characters that you think would be of concern, you should raise them as issues.<br><br></font>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<font size="2"><br><br> > 5. The character does not have the Script value: Hangul<br></font> <div><font size="2">><br> > The REMAP characters are removed from DISALLOWED, so that the<br> > TABLES values form a partition (all the values are disjoint).<br>
<br></font>
</div><font size="2">This strikes me as dangerous -- see below.<br><br> > B. Protocols documentChange sections 4.2.1 and 5.3 so as to<br> > require:<br> ><br> > 1. Mapping all REMAP characters according to the Unicode<br>
> property NFKC_Casefold,<br> > 2. Then normalizing the result according to NFC.<br><br> Making this change to 4.2.1 eliminates the requirement that the<br> registrant understand _exactly_ what is being registered, i.e.,<br>
that the communication path between the registrant and registry<br> occur only using U-labels and/or A-labels. My understanding was<br> that we had reached one of the more clear consensus we had in<br> these discussions that the "no mapping on registration"<br>
restriction was appropriate. Are you proposing to reopen that<br> question?</font> </blockquote><div><font size="2"><br>Sorry, you are correct. This would only affect the lookup part.<br> <br></font></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<font size="2"><br></font> <div><font size="2"><br> > The rest of the tests for U-Label remain unchanged.<br><br></font> </div><font size="2">I believe that doing this by the type of change to Tables that<br> you recommend either requires a change to the way that the<br>
definition of U-label is stated or requires us to abandon the<br>
very clear concept of a U-label that is completely symmetric,<br> with no information loss in either direction, with an A-label.</font></blockquote><div><font size="2"><br>I don't see why you would think that. A U-Label remains just the way it is, and has a 1-1 relation with an A-Label. The only difference is that we have an additional category of M-Label; one that is not a U-Label but maps to one.<br>
<br></font>
</div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><font size="2"><br><br> There is also a subtle interaction with Section 5.5: if the<br>
mapping is performed by the time Section 5.3 concludes (or,<br>
under special circumstances, not applied at all), then Section<br> 5.5 must also prohibit REMAP.</font> </blockquote><div><font size="2"><br>You are correct; that was my intention, but I forgot to mention it. Yes, there needs to be a change in 5.5.<br>
<br>So below:<br></font>
<pre><font size="2"> o Labels containing prohibited code points, i.e., those that are<br><br> table [<a href="http://tools.ietf.org/html/draft-ietf-idnabis-protocol-12#ref-IDNA2008-Tables" title=""The Unicode Codepoints and IDNA"" target="_blank">IDNA2008-Tables</a>].<br>
</font>
assigned to the "DISALLOWED" category in the permitted character
</pre><font size="2"> add<br></font><pre><font size="2"> o Labels containing remapped code points, i.e., those that are<br> assigned to the "REMAP" category in the permitted character<br> table [<a href="http://tools.ietf.org/html/draft-ietf-idnabis-protocol-12#ref-IDNA2008-Tables" title=""The Unicode Codepoints and IDNA"" target="_blank">IDNA2008-Tables</a>].<br>
</font>
</pre><font size="2"><br></font></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><font size="2"><br><br> > C. Defs document<br> ><br>
> 1. Define REMAP<br> > 2. Define an M-Label to be one which if remapped according<br></font>
<div><font size="2">> to B1+B2, results in a U-Label.<br><br></font> </div><font size="2">The idea of an M-Label still makes me uncomfortable. Again, we<br> have had that discussion before.<br><br> regards,<br></font> <font color="#888888" size="2"> john<br>
</font><div>
<div><font size="2"><br><br><br><br> ______________________________</font> <font size="2">_________________<br> Idna-update mailing list<br><a href="mailto:Idna-update@alvestrand.no" target="_blank">Idna-update@alvestrand.no</a><br>
<a href="http://www.alvestrand.no/mailman/listinfo/idna-update" target="_blank">http://www.alvestrand.no/mailman/listinfo/idna-update</a><br></font>
</div></div></blockquote></div><font size="2"><br> ______________________________</font><font size="2">_________________<br>Idna-update mailing list<br><a href="mailto:Idna-update@alvestrand.no" target="_blank">Idna-update@alvestrand.no</a><br>
<a href="http://www.alvestrand.no/mailman/listinfo/idna-update" target="_blank">http://www.alvestrand.no/mailman/listinfo/idna-update</a><br></font>
</blockquote></div><font size="2"><br></font></div></div></div></div><font size="2"><br>______________________________</font><font size="2">_________________<br>
Idna-update mailing list<br><a href="mailto:Idna-update@alvestrand.no" target="_blank">Idna-update@alvestrand.no</a><br><a href="http://www.alvestrand.no/mailman/listinfo/idna-update" target="_blank">http://www.alvestrand.no/mailman/listinfo/idna-update</a><br>
<br></font>
</blockquote></div><font size="2"><br></font>