<span style="font-family: arial,helvetica,sans-serif;">You're saying you don't want "automatic incorporation of changes initiated by the Unicode community". But what Ken is proposing is precisely the reverse: it is *preventing* automatic incorporation of changes in pre-existing characters initiated by the Unicode community.<br>
<br>Having automatic generation of the BackwardCompatible table would ensure that changes to pre-existing characters from Unicode would *not* automatically be picked up by IDNA. It insulates IDNA from any Unicode changes to previously assigned characters. If we *don't* put into place something like what Ken proposes, we *do* get </span><span style="font-family: arial,helvetica,sans-serif;">"automatic incorporation of changes initiated by the Unicode community".<br>
<br>This may seem paradoxical, but</span> let's look at how the situation plays out with a concrete example.<br><br>Suppose that X is a letter and Y is a symbol in Unicode 5.1. The Unicode consortium receives information that X is actually a symbol and Y is a letter, and makes that change in Unicode 5.2 next fall. And let's say half of the implementations of IDNA (both lookup clients and registries) update to Unicode 5.2 when released and half don't. (In reality implementations will update in a much more haphazard way, in accordance with whatever their ship schedule is. Google's search engine like Google typically updates almost immediately, while a browser sitting on someone's machine might not be updated for months, or years. But for the sake of illustration, let's say that this is half and half.)<br>
<div><br></div><div>Scenario A. BackwardCompatible *is not* automatically generated by IANA.<br></div><div><ol><li>The updated implementations dutifully run the algorithm in Tables, and now consider X to be DISALLOWED, and Y to be PVALID.</li>
<li>The unupdated implementations still consider X to be PVALID and Y to be DISALLOWED<br></li><li>So updated registries now disallow their registrations with X (refunding people's money?), and allow registrations with Y. And updated lookup clients will refuse to lookup X, and will lookup Y.<br>
</li><li>Unupdated registries still allow registrations with X, and disallow registrations with Y. And unupdated lookup clients will allow lookups with X, and will refuse to lookup Y.</li><li>There will be obvious conflicts among mismatched implementations and registries; and these conflicts are not the result of new characters; they are among characters that all of the implementations are meant to handle.<br>
</li><li>Then the IETF decides this is a bad situation, starts the whole process of doing a new RFC. Some time later, this adds X and Y to BackwardCompatible to restore the previous situation.<br></li><li>Some subset of the updated implemenations now pull in the new BackwardCompatible list, and now they revert to the 5.1 behavior for these. Others don't update to that table immediately, and still conflict with unupdated implemenations until they are updated.</li>
<li>This is a bit of a mess, and an avoidable one.<br></li></ol>Scenario B. BackwardCompatible *is* automatically generated by IANA. (Ken's proposal).<br><ol><li>The implemenations that update to Unicode 5.2 dutifully run the algorithm in Tables, with the BackwardCompatible table generated for Unicode 5.2.<br>
</li><li>Because IANA has updated BackwardCompatible, they won't change X and Y from what they were in Unicode 5.1.</li><li>No compatibility problems for X and Y.<br></li></ol>Scenario B does *not* preclude the IETF from considering the situation with X and Y, and deciding to go with all or part of the Unicode 5.2 change (although I'd recommend against it!). They could then issue a new RFC that changes the ExceptionTable so as to add X and Y (or just one, if desired). Such a step would still cause some problems with incompatible implementations, but the release of the RFC would be at the IETFs pace, the compatibility problems would be known (and worth the price according to the IETF, or it wouldn't do it), and we wouldn't get the back and forth instabilities where a character goes from PVALID to DISALLOWED and back, or the reverse.<br>
<br>Having these kinds of procedures are nothing new for the IETF. BCP47, for example, has had this kind of process in place. That is, in the specification it has precise rules for how to update tables based on what its source standards (ISO in this case) do, so as to maintain stability.<br>
<br>Now, I do think that the likelyhood of the X/Y case happening is quite small, and essentially zero with any frequently used characters. Clearly "A" is not going to change to a Symbol! But the likelyhood is not zero. For the small number of exceptions that we've had to a similar backward-compatibility table for Unicode identifiers (over the span of many versions), see <a href="http://www.unicode.org/reports/tr31/#Backward_Compatibility">http://www.unicode.org/reports/tr31/#Backward_Compatibility</a>.<br>
<br>But as long as we are devising this process, the changes that Ken proposes in order to make it completely bullet-proof are small, and give us the security that no matter what happens in future Unicode versions, stability of IDNs is guaranteed even without further action by the IETF.<br>
<div><br clear="all">Mark<br>
<br><br><div class="gmail_quote">On Tue, Dec 9, 2008 at 18:25, Erik van der Poel <span dir="ltr"><<a href="mailto:erikv@google.com" target="_blank">erikv@google.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
Martin and Ken,<br>
<br>
I believe the point is that *if* a relevant Unicode property changes,<br>
no matter how unlikely, then the IETF and other DNS stakeholders ought<br>
to get a chance to consider whether or not to make the consequent<br>
incompatible change to IDNA. After all, if the Unicode community<br>
decided that the change was important enough to make, then the DNS<br>
community may also decide that it is important enough to change on the<br>
IDNA side.<br>
<br>
So, we do not want automatic incorporation of changes initiated by the<br>
Unicode community. Instead, we want the DNS community to make the<br>
right decision for DNS, which is something the Unicode community is<br>
not qualified to do.<br>
<br>
Just look at what happens when IDNA blindly adopts Unicode specs: The<br>
eszett/ss debacle. Now we have to make a painful change for that,<br>
because the German registry insists that eszett be included. Of<br>
course, the German registry should have voiced their opinion when<br>
IDNA2003 was being drafted, but...<br>
<br>
So, we need a group of experts to first try the automatic derivation,<br>
and then see whether there are any PVALID->DISALLOWED or<br>
DISALLOWED->PVALID changes. If there are, then the issue must be<br>
discussed, probably via Internet Drafts, possibly without restarting<br>
the WG. Then each character must be put into BackwardCompatible or<br>
Exceptions, depending on the decision. (Or it might be simpler to<br>
always put such characters into Exceptions, and just not have a<br>
BackwardCompatible category.)<br>
<font color="#888888"><br>
Erik<br>
</font><div><div></div><div><br>
On Tue, Dec 9, 2008 at 5:52 PM, Martin Duerst <<a href="mailto:duerst@it.aoyama.ac.jp" target="_blank">duerst@it.aoyama.ac.jp</a>> wrote:<br>
> But isn't this exactly what we do NOT want? If I understand<br>
> correctly, BackwardCompatible is a purely administrative thing,<br>
> ideally we give IANA an algorithm, and they execute it and<br>
> put the result into the registry when there is an update<br>
> to Unicode properties that requires a balancing entry in<br>
> BackwardsCompatibile. Bothering the IETF at large with this<br>
> is pretty useless; having an expert reviewer as a "goto guy"<br>
> for IANA will probably help.<br>
><br>
> On the other hand, for Context, changes are usuall something<br>
> new and unexpected, and potentially even political. A somewhat<br>
> more serious/heavy process seems appropriate.<br>
</div></div></blockquote></div><br></div>
</div>