<br><br><div><span class="gmail_quote">On 3/15/07, <b class="gmail_sendername">John C Klensin</b> <<a href="mailto:klensin@jck.com">klensin@jck.com</a>> wrote:</span><div><br>[snip] <br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
I'm trying to understand this experiment. Normally, an href<br>that "uses IDNA" would have Punycode labels (A-labels) in its<br>domain names. </blockquote><div><br>I don't know the basis for saying that this would be the "normal" usage. There isn't anything in IDNA2003, unless I'm missing something, that requires or even suggests that it is not perfectly fine to have:
<br><br><a href="<a href="http://ÖBB.at">http://ÖBB.at</a>">Österreichishe Bundesbahn</a><span></span></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
If that were the case, presumably at least most<br>of the transformations you are describing below would be<br>non-issues since the A-labels are already in reduced form, with<br>all case variations forced to lower, all compatibility
<br>characters reduced to canonical form, etc.</blockquote><div><br>True. Any pages that are already in Punycode should already be ok. <br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
So, presumably, if you are running ToUnicode against the<br>contents of hrefs, you are looking at hrefs that either use<br>UTF-8 (or some other encoding) directly as domain names (a<br>string of U-labels in IDNA200x terminology) --
i.e., are IRIs<br>rather than URIs -- or contain the UTF-8 strings in %-escape<br>form. But the last I checked, the latter was not a recommended<br>practice for domain names (A-labels are generally considered<br>preferable if real U-labels cannot be used)
</blockquote><div><br>(above)<br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">and support for IRIs<br>was not widespread in the installed browser base (which of
<br>course contains many copies of versions of IE prior to IE7,<br>etc.). The latter suggests that those who are interested in<br>having their web pages accessible from a large number of<br>browsers are probably not using IRIs yet.
</blockquote><div><br>What is what I would have thought; I was surprised by the data as well. I can only surmise that there was sufficient drive to use IDNA that people did either use the browsers that handled IDNA natively or downloaded the IE plugin that enabled it.
<br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">Am I missing something or, if not, could you summarize where<br>these hrefs are coming from? Even though the IDNA-containing
<br>hrefs appear to constitute less than 0.2% of the hrefs you<br>examined (it would be around 0.2% if those billion documents<br>contains only one href each), my intuition suggests that it<br>might be somewhat more than I expected.
</blockquote><div><br>These are from a random sampling of pages in Google's index, where each of the pages was scanned for href's that contained IDNAs. Of course, a billion pages is a small sampling of the web....
<br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">You also asked...<br><br>> Actually, one question that has come up. It appears that in
<br>> <a href="http://www.ietf.org/internet-drafts/draft-klensin-idnabis-issu">http://www.ietf.org/internet-drafts/draft-klensin-idnabis-issu</a><br>> es-01.txt no mappings are being done, thus the "B.1 Commonly
<br>> mapped to nothing" characters from rfc3454 are simply illegal.<br>> The only ones that would be mapped to nothing would be the<br>> joiners (subject to context).<br>><br>> Is this the intent?<br>
<br>Yes. I think it was the intent that these be prohibited even in<br>IDNA2003 although our collective understanding might not have<br>been sufficient to get things right at the time. One way of<br>looking at this is that, regardless of whatever measures are
<br>taken, one of the most important weapons against either general<br>confusion or malicious acts (such as phishing), user intuition<br>as to whether or not two domain name strings are the same, based<br>on visual inspection of those strings, should mostly get the
<br>right answer with very little astonishment. </blockquote><div><br>I agree -- see below on case and width folding. <br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Naturally, we can<br>expect more surprises with scripts that are unfamiliar to the<br>user than with familiar ones, and visual comparison of domain<br>names tells us nothing about the values in the underlying<br>resource records and where they point, but applying restrictions
<br>to reduce obvious sources of such confusion or astonishment<br>appears to be generally a good idea, at least in the absence of<br>arguments for particular code points that are sufficient to<br>overwhelm the downside risks.
<br><br>In that regard, invisible characters and characters that are<br>visible but later disappear, are the friends of those who want<br>to create confusion since they can make two strings that appear<br>different visually to be the same or two strings that appear the
<br>same visually to be different.</blockquote><div><br>The chief problem is where two different strings have the same visual representation -- (<a href="http://paypal.com">paypal.com</a>), not the case where the same string has two different visuals. It is not much of a problem that, say, both a fullwidth A and a normal A are treated the same, nor that a lowercase and uppercase A are treated the same; in fact, all but software engineers expect them to be treated the same. It is where case/width differences are treated as significant that average people get confused.
<br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">As with anything else that the IDNA200X model prohibits from<br>appearing in a U-label, nothing prevents an properly localized
<br>implementation from accepting characters that are banned by the<br>proptocol and mapping them as appears reasonable under local<br>conditions.</blockquote><div><br>I'm very leery of that statement. I think it is a really bad idea that browser A could say treat <a href="
<a href="http://Μαρκ.com">Μαρκ.com</a>">....</a> as if it were<br><a href="<a href="http://Mark.com">Mark.com</a>">....</a> and browser B could treat <a href="<a href="http://Μαρκ.com">
Μαρκ.com</a>">....</a> as if it were <a href="<a href="http://μαρκ.com">μαρκ.com</a>">....</a>, and both correctly claim conformance to IDNA2003. That seems to me a huge security hole, as well as in practice a nightmare.
<br></div><br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"> john<br><br>p.s. I owe responses for a number of other notes that have been
<br>posted to this list. I got hit by several pre-IETF priority<br>demands on my time and will try to dig out over the next few<br>days.<br><br></blockquote></div><br><br clear="all"><br>-- <br>Mark