Catching up on IDNA.... Comments below on stability.<br><br><div class="gmail_quote">On Dec 25, 2007 2:39 AM, Patrik Fältström <<a href="mailto:patrik@frobbit.se">patrik@frobbit.se</a>> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div style=""><div><div class="Ih2E3d"><div>On 17 dec 2007, at 22.39, Mark Davis wrote:</div><br></div></div><div><div class="Ih2E3d"><div><br></div></div>What we have said is that the properties we use in the algorithms have to be stable.
</div></div></blockquote><div><br>I think this is the crux of the issue. The goal is for the <b>IDNA property</b> to be stable. Bear with me for a minute -- the following may seem to be pedantic, but I am just trying to take this slowly so that I don't miscommunicate.
<br><br>It appears that we have a case of affirming the consequent, with the following argument:<br><br><ol><li>If the base properties are stable, then the IDNA property will be stable. [A => B]<br></li><li>The IDNA property must be stable. [B]
<br></li><li>Therefore the base properties must be stable. [A]<br></li></ol><br>However, that argument is invalid, and the conclusion isn't true. Take a parallel case. Suppose we have a room with a fixed number of people in it, including me and Bill Gates. Someone has a condition that the average salary be stable, and argues:
<br><br><ol><li>If each individual's salary is stable, then the average salary will be stable.</li><li>The average salary must be stable.</li><li>Therefore the individual's salaries must be stable.</li></ol><br>But the argument is invalid. Suppose Bill pays me a billion dollars. Our individual salaries changed radically, yet the average remains the same. (Just to be clear on this, Michael, in the interests of progressing IDNA, I am willing to try this experiment out ...)
<br><br>Let's get back to the overall goal, which is stability of the IDNA property. Many changes in base properties have no affect. For example, if a character changes the script property,
but from a script that's included to another script that's included, it
makes no difference in the outcome of the rules. There is the possibility of a base property changing, in a new version of Unicode, so as to affect the results according to your current formulation. But as pointed out, there is an easy and reliable way to modify your formulation so as to be perfectly stable.
<br><br><div style="margin-left: 40px;">Define the following contributing properties, initially empty. Make the rules for ALWAYS and NEVER always include them (and exclude the converse ones).<br><br><ul><li>ALWAYS_INCLUSIONS
</li><li>NEVER_INCLUSIONS</li></ul><br>With each version X of Unicode, update them so to include any characters that were in ALWAYS or NEVER according to version X-1, but wouldn't be according to X without these properties.
<br></div><br>The updates could be best be done by the Unicode consortium as a separate so-called contributing property, but it is perfectly possible for any other organization to do it independently if you're nervous about that. As discussed in earlier email, we've been doing this kind of thing for years with the current Unicode identifiers, and there are a very small number of characters affected. Note some work needs to be done with each version of Unicode anyway, since as new scripts come in, they have to be put in one or another buckets.
<br><br>...<br><br>Given that, let me take a shot at answering your questions.<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div style="">The questions now I think are:<div><br></div><div>a. What differences exist between the algorithms in the table document and Unicode "stable" properties?</div></div></blockquote><div>Some of the properties are stable, like isNFKC, and some are not: general category, script,...
<br> <br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div style=""><div></div><div>b. What differences exists between IDNA2003 and the tables document?
</div></div></blockquote><div><br>Well, the easiest way to play with that is to use the utilities I mentioned.<br><br>Go to <a href="http://unicode.org/cldr/utility/unicodeset.jsp">http://unicode.org/cldr/utility/unicodeset.jsp
</a> and in Input A, put:<br><br><div style="margin-left: 40px;">[:idna=output:]<br><br></div>and in Input B, put<br><br><div style="margin-left: 40px;">[[:L:][:Nd:][:Mn:][:Mc:]<br>&[:isCaseFolded:]<br>-[:NFKC_QuickCheck=NO:]
<br>-[:Default_Ignorable_Code_Point:]<br> [\u00B7\u05F3\u05F4\u3007\u30FB]<br> [a-zA-Z0-9\-]<br>&[:age=3.2:]]<br></div><br>Then hit Compare.<br><br>This doesn't include the script, blocks and exceptions, but will give you an idea of the difference. Note that I put in the condition that the Age (that is, Unicode Version, be
3.2, so that we'd be comparing Apples to Apples.<br><br>If you click on one of the links, you see the differences in more detail. In this case, for example, the characters in IDNA2003 (output) that are not in the formulation are 3,032 Code Points.
<br><br>If you are feeling more adventurous, you can exclude scripts/blocks according to the draft Unicode 5.1 recommendations (see Ken's emails, plus <a href="http://www.unicode.org/reports/tr31/tr31-8.html#Specific_Character_Adjustments">
http://www.unicode.org/reports/tr31/tr31-8.html#Specific_Character_Adjustments</a>).<br><br>That expression for Input B would be:<br><br>[[:L:][:Nd:][:Mn:][:Mc:]<br> &[:isCaseFolded:]<br> -[:NFKC_QuickCheck=NO:]<br> -[:Default_Ignorable_Code_Point:]
<br> -[:script=Bugi:]<br> -[:script=Buhd:]<br> -[:script=Cari:]<br> -[:script=Copt:]<br> -[:script=Cprt:]<br> -[:script=Dsrt:]<br> -[:script=Glag:]<br> -[:script=Goth:]<br> -[:script=Hano:]<br> -[:script=Ital:]<br> -[:script=Khar:]
<br> -[:script=Linb:]<br> -[:script=Lyci:]<br> -[:script=Lydi:]<br> -[:script=Ogam:]<br> -[:script=Osma:]<br> -[:script=Phag:]<br> -[:script=Phnx:]<br> -[:script=Rjng:]<br> -[:script=Runr:]<br> -[:script=Shaw:]<br> -[:script=Sund:]
<br> -[:script=Sylo:]<br> -[:script=Syrc:]<br> -[:script=Tagb:]<br> -[:script=Tglg:]<br> -[:script=Ugar:]<br> -[:script=Xpeo:]<br> -[:script=Xsux:]<br> -[:block=Combining_Diacritical_Marks_for_Symbols:]<br> -[:block=Musical_Symbols:]
<br> -[:block=Ancient_Greek_Musical_Notation:]<br> [\u00B7\u05F3\u05F4\u3007\u30FB]<br> [a-zA-Z0-9\-]<br> &[:age=3.2:]] <br><br>It shows that there would be 3,419 Code Points in IDNA2003 output that are not in that formulation. Click on the link for details. I did remove the Phaistos Disk block, since that is not yet in Unicode and thus not recognized by the utilities.
<br><br>Hope that helps...<br></div><div> </div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;"><div style=""><div></div><div>c. What differences will exist if tables document algorithms are only based on stable properties (where some of them are only stable from Unicode
5.0)</div></div></blockquote><div> </div><div>The general category is crucial. However, as stated above, we don't need this for stability anyway.<br><br></div><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<div style=""><div></div><font color="#888888"><div><br></div><div> Patrik</div><div><br></div></font></div></blockquote></div><br><br clear="all"><br>-- <br>Mark