I'm afraid I was a bit too terse (failing of mine that other Unicoders will atest to). Let me try to flesh it out.<br><br>When you case-fold in Unicode, you remove certain distinctions. That is, you have cases where, say, 3 characters map to one. Because there are a very small number of source characters that could map to any target character, as you do the folding you can also extract a set of bits that tell you which of the original characters was mapped. That vector of bits, plus the target string, can get you back to the original casing. You need no bits for uncased characters (eg Chinese), need typically 1 bit per cased character, and in exceptional cases need more than one bit. (For σ, Σ, ς you'd have 00, 01, 10, for example.)<br>
<br>The simplest mechanism would be to then take that set of bits and walk through the Punycode, and for each bit in the vector changing each cased letter to uppercase to represent a 1 bit, and leaving it lowercase represent a 0 bit.<br>
<br>You then end up with an A-label that contains enough information that it can be converted back to the cased version of the U-label, yet a case-insensitive comparison of A-labels yields the same result as a case-insensitive comparison of the original U-labels.<br>
<br>Thus Σωτήρης would convert to xn--jxas3agdc8a in Punycode. The bit vector for conversion back would be <01,0,0,0,0,0,10>. Applying the casing bits we'd get xN--jxas3aGdc8a.<br><br>As another example, Oréal would convert to xn--oral-cpa, with a bit vector of <1,0...>, thus yielding Xn--oral-cpa.<br>
<br>This would work for any label where you don't run out of ASCII letters before you run out of one bits. I haven't done any comprehensive tests, but I would suspect that we could preserve case in the vast majority of cases just with this simple mechanism. And as I think about it, I strongly suspect that we could apply certain compression techniques to cover essentially all cases, and end up with a mechanism that would actually be backwards compatible that preserved input Unicode. (Cc'ing Markus, who might be interested in the bit-twiddling aspect.)<br>
<br>I hope that's a bit clearer.<br><br>Mark<br><br><div class="gmail_quote">On Sat, Mar 29, 2008 at 9:03 AM, John C Klensin <<a href="mailto:klensin@jck.com">klensin@jck.com</a>> wrote:<br><blockquote class="gmail_quote" style="border-left: 1px solid rgb(204, 204, 204); margin: 0pt 0pt 0pt 0.8ex; padding-left: 1ex;">
<br>
<br>
--On Saturday, March 29, 2008 8:47 AM -0700 Mark Davis<br>
<div class="Ih2E3d"><<a href="mailto:mark.davis@icu-project.org">mark.davis@icu-project.org</a>> wrote:<br>
<br>
> Patrik, you misunderstand. I'm not saying that this should be<br>
> part of the protocol. What I'm saying is that the protocol,<br>
> *combined with a postprocessing step for UIs, *would handle<br>
> the situation.<br>
><br>
> (In retrospect -- and water long under the bridge -- we would<br>
> have been better off to use one of the variants of Punycode,<br>
> which has the ability to encode case and other distinguishing<br>
> information in the original Unicode using case in the ASCII<br>
> form. Had we gone that route, we could have maintained the<br>
> visual distinctions on output of DNS for sigma and similar<br>
> cases, because the DNS does a caseless compare for A-Z.)<br>
<br>
</div>Mark,<br>
<br>
Unless I misunderstand what you are suggesting, that punycode<br>
variation would not have helped. Because the code points are<br>
different, punycode(raw-upper-case-string) is not going to<br>
contain the same characters as<br>
punycode(equivalent-lower-case-string). One could use punycode<br>
case to encode things the way you suggest, but only by case<br>
folding first and then using the punycode case to indicate "used<br>
to be upper case". But that wouldn't help for the sigma<br>
situation because the case folding operation itself is what<br>
loses the information (about final form, not really case), and<br>
that isn't subject to a binary "upper/lower" switch.<br>
<br>
Or have I missed something in what you are suggesting?<br>
<br>
But, one way or the other, certainly water under the bridge.<br>
<font color="#888888"><br>
john<br>
<br>
<br>
<br>
</font></blockquote></div><br><br clear="all"><br>-- <br>Mark