<html><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space; ">Mark, et al,<div><br></div><div>broad re-mapping, even if only for lookup, makes me wonder about ways to mislead users through the use of labels in domain names that would not be allowed until mapped. I think I can understand how we might want NOT to map some of the recently introduced characters (sharp-s for instance) and how we might want to map upper case into lower case prior to lookup (emulating the case independent matching of the purely ASCII domain names/labels of the past. I am having some difficulty with the full range of characters that might be invalid under IDNA2008 but mapped into valid IDNA2008 characters. If there has been a trend in the discussions it has been towards limiting the set of characters that may be mapped prior to lookup. </div><div><br></div><div>I think we need to find a space around which compromise and consensus can built as to what chars are allowed to be mapped prior to look up. I think we all agree that there should be no implicit or explicit mapping in the registration process. </div><div><br></div><div>Looking for more common ground.</div><div><br></div><div>vint</div><div><br></div><div><br><div><div>On Jun 29, 2009, at 3:46 PM, Mark Davis ⌛ wrote:</div><br class="Apple-interchange-newline"><blockquote type="cite"><br clear="all">Mark<br> <br><br><div class="gmail_quote">On Mon, Jun 29, 2009 at 10:55, John C Klensin <span dir="ltr"><<a href="mailto:klensin@jck.com">klensin@jck.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> Mark,<br> <br> Several comments inline...<br> <br> --On Sunday, June 28, 2009 21:28 -0700 Mark Davis ⌛<br> <div class="im"><<a href="mailto:mark@macchiato.com">mark@macchiato.com</a>> wrote:<br> <br> > Returning to the discussion, now that some of my other<br> > standards work is under control (RFC4646bis was approved,<br> > whew!)<br> </div>>...<br> <div class="im"><br> > Now, my position is still that the simplest and most<br> > compatible option open to us is to simply map with NFKC +<br> > Casefold.<br> <br> </div>I continue to believe that CaseFold is a showstopper. When its<br> results are not identical to those produced by LowerCase, it<br> produces results that are astonishing to some users and leads us<br> into the "is that a separate character or not" trap that we've<br> seen manifested at least twice. I note that TUS recommends<br> against its use for mapping (as distinct from comparison) and<br> appears to do so for just the reason that it involves too much<br> information loss.</blockquote><div><br>You need to provide actual data behind this. Please list exactly the characters that you mean, and why you think they are problematic. Note also that the formulation that I gave means that any character that is PVALID would automatically be excluded, eg if final-sigma is PVALID then it is unaffected. And we can certainly introduce other exceptions.<br> <br>And I know full well about the issues in TUS, having written or participated in the writing of them.<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> >...<br> <div class="im">> Proposal: A. Tables document<br> ><br> > Add a new type of character: REMAP. A character is REMAP if it<br> > meets *all of * the following criteria:<br> ><br> </div>> 1. The character is not PVALID or CONTEXTO<br> > 2. If remapped by the Unicode property NFKC_Casefold*, then<br> <div class="im">> the resulting character(s) are all PVALID or CONTEXTO<br> </div>> 3. The character is a LetterDigit or Pd<br> > 4. The character has one of the following<br> <div class="im">> Decomposition_Type values: initial, medial, final,<br> > isolated, wide, narrow, or compat<br> <br> </div>I am very concerned that collapsing initial, medial, and final<br> together may get us into problems with other language<br> communities similar to those we have gotten into with Final<br> Sigma, especially when those communities denote word boundaries<br> by the appearance of final or initial forms and hence would use<br> those forms in a style similar to the way "BigCompany" or<br> "big-company" might be used in ASCII.</blockquote><div><br>The mechanism used to indicate boundaries is not, as you think, the use of the presentation forms; it is the use of the ZWNJ/J, which we already provide for.<br> <br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><br> <br> As I've said several times before, even if we disallow the<br> NFKC-affected forms those characters, if a need arises, we can<br> (painfully) redefine them as PVALID and allow them. But, if we<br> map them to something else, we lose all information about what<br> was intended/desired and end up in precisely the mess we have<br> with e.g., Final Sigma and ZWJ/ZWNJ in which "the right thing<br> to do" poses enough compatibility problems to cause opposition<br> to making changes.</blockquote><div><br>You make it sounds like final sigma, ZWJ/NJ, eszett and the other cases under discussion were oversights in the process of developing the current IDNA. That wasn't the case; these were deliberate choices made at the time. A case mapping is also a 'loss of information', but one that people clearly want.<br> <br>If you have any particular characters that you think would be of concern, you should raise them as issues.<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> <br> <br> > 5. The character does not have the Script value: Hangul<br> <div class="im">><br> > The REMAP characters are removed from DISALLOWED, so that the<br> > TABLES values form a partition (all the values are disjoint).<br> <br> </div>This strikes me as dangerous -- see below.<br> <br> > B. Protocols documentChange sections 4.2.1 and 5.3 so as to<br> > require:<br> ><br> > 1. Mapping all REMAP characters according to the Unicode<br> > property NFKC_Casefold,<br> > 2. Then normalizing the result according to NFC.<br> <br> Making this change to 4.2.1 eliminates the requirement that the<br> registrant understand _exactly_ what is being registered, i.e.,<br> that the communication path between the registrant and registry<br> occur only using U-labels and/or A-labels. My understanding was<br> that we had reached one of the more clear consensus we had in<br> these discussions that the "no mapping on registration"<br> restriction was appropriate. Are you proposing to reopen that<br> question?</blockquote><div><br>Sorry, you are correct. This would only affect the lookup part.<br> <br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> <br> <div class="im"><br> > The rest of the tests for U-Label remain unchanged.<br> <br> </div>I believe that doing this by the type of change to Tables that<br> you recommend either requires a change to the way that the<br> definition of U-label is stated or requires us to abandon the<br> very clear concept of a U-label that is completely symmetric,<br> with no information loss in either direction, with an A-label.</blockquote><div><br>I don't see why you would think that. A U-Label remains just the way it is, and has a 1-1 relation with an A-Label. The only difference is that we have an additional category of M-Label; one that is not a U-Label but maps to one.<br> <br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><br> <br> There is also a subtle interaction with Section 5.5: if the<br> mapping is performed by the time Section 5.3 concludes (or,<br> under special circumstances, not applied at all), then Section<br> 5.5 must also prohibit REMAP.</blockquote><div><br>You are correct; that was my intention, but I forgot to mention it. Yes, there needs to be a change in 5.5.<br><br>So below:<br><pre class="newpage"> o Labels containing prohibited code points, i.e., those that are<br>
assigned to the "DISALLOWED" category in the permitted character<br> table [<a href="http://tools.ietf.org/html/draft-ietf-idnabis-protocol-12#ref-IDNA2008-Tables" title=""The Unicode Codepoints and IDNA"">IDNA2008-Tables</a>].<br>
</pre> add<br><pre class="newpage"> o Labels containing remapped code points, i.e., those that are<br> assigned to the "REMAP" category in the permitted character<br> table [<a href="http://tools.ietf.org/html/draft-ietf-idnabis-protocol-12#ref-IDNA2008-Tables" title=""The Unicode Codepoints and IDNA"">IDNA2008-Tables</a>].<br>
</pre><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><br> <br> > C. Defs document<br> ><br> > 1. Define REMAP<br> > 2. Define an M-Label to be one which if remapped according<br> <div class="im">> to B1+B2, results in a U-Label.<br> <br> </div>The idea of an M-Label still makes me uncomfortable. Again, we<br> have had that discussion before.<br> <br> regards,<br> <font color="#888888"> john<br> </font><div><div></div><div class="h5"><br> <br> <br> <br> _______________________________________________<br> Idna-update mailing list<br> <a href="mailto:Idna-update@alvestrand.no">Idna-update@alvestrand.no</a><br> <a href="http://www.alvestrand.no/mailman/listinfo/idna-update" target="_blank">http://www.alvestrand.no/mailman/listinfo/idna-update</a><br> </div></div></blockquote></div><br> _______________________________________________<br>Idna-update mailing list<br><a href="mailto:Idna-update@alvestrand.no">Idna-update@alvestrand.no</a><br>http://www.alvestrand.no/mailman/listinfo/idna-update<br></blockquote></div><br></div></body></html>