The real issue: interopability, and a proposal (Was: Consensus Call on Latin Sharp S and Greek Final Sigma)

[Tina Dam]

Hi Mark, I respectfully disagree with your assessment. Also based on the correspondence I have received over the last several years, explains another reasons that Greek and other non-Latin based addresses are not used as much. It is simply quite inconvenient to have IDNs at the second level in non-Latin based scripts, then to switch script when typing the top level portion of the address.

For that reason IDNs at the second level in certain scripts has not been introduced or if introduced then not considered useful.

This is anticipated to change as we get the IDN TLDs launched.

Tina

From: idna-update-bounces at alvestrand.no [mailto:idna-update-bounces at alvestrand.no] On Behalf Of Mark Davis ?
Sent: Tuesday, December 01, 2009 3:23 PM
To: Erik van der Poel
Cc: Shawn Steele; Patrik Fältström; Harald Alvestrand; idna-update at alvestrand.no; lisa Dusseault; Alexander Mayrhofer; Martin J. Dürst; Vint Cerf
Subject: Re: The real issue: interopability, and a proposal (Was: Consensus Call on Latin Sharp S and Greek Final Sigma)

One addition: there is over 35 times as much German content as Greek, so that explains part of the difference in final-sigma vs eszed proportion. (The relative proportion per language is important.)

Mark

2009/12/1 Erik van der Poel <erikv at google.com<mailto:erikv at google.com>>
I ran the program again today, and Eszett is being used a bit more now
than it was last year.

2009-11-28
1,253,099,703 documents
88,712,912,831 links
8,981 Eszett in host name in link 0.00001%
furz-großerfurz.de<http://furz-grosserfurz.de>
www.bußgeldexperten.de<http://www.bussgeldexperten.de>
www.metzgerei-gaßner.de<http://www.metzgerei-gassner.de>

2008-11-19
819,600,672 documents
49,904,513,188 links
2,739 Eszett in host name in link 0.0000055%
www.rtc-großefehn.de<http://www.rtc-grossefehn.de>
www.mein-fußballclub.de<http://www.mein-fussballclub.de>
www.dermaßanzug.com<http://www.dermassanzug.com>

2006-11-27
889,759,121 documents
1,973 Eszett in host name in document URL 0.00022%
www.uni-gießen.de<http://www.uni-giessen.de>
www.uni-gießen.de<http://www.uni-giessen.de>
www.uni-gießen.de<http://www.uni-giessen.de>

All 3 of the samples were "high value" documents in our index. The
2006 sample looked for Eszett in the host name in the URL of the
document (rather than links inside the document). It is no longer
possible to find Eszett in the URLs of our documents because they are
now all mapped to "ss". So the 2006 sample cannot really be compared
with the others because the URL of a document always contains a host
name, while a link inside a document might be a relative URL (without
a host name).

The Final Sigma has not grown as much:

2009-11-28
305 final sigma in host name in link
0.00000034%
www.γυναικολόγος.gr<http://www.xn--mxadbxfgktc4bn4g.gr>
www.γυναικολόγος.gr<http://www.xn--mxadbxfgktc4bn4g.gr>
www.γυναικολόγος.gr<http://www.xn--mxadbxfgktc4bn4g.gr>

2008-11-19
138 final sigma in host name in link
0.00000028%
www.ταβερνες.gr<http://www.xn--mxacja3bxaqb.gr>
www.ελληναΐς.gr<http://www.xn--owa9dlitap4c.gr>
www.γυναικολόγος.gr<http://www.xn--mxadbxfgktc4bn4g.gr>

Erik

On Tue, Dec 1, 2009 at 11:49 AM, Mark Davis ☕ <mark at macchiato.com<mailto:mark at macchiato.com>> wrote:
> It is approximately 60, as you computed. The trillion figure was in a public
> posting from July 2008, which is why we can quote it.
>
> Mark
>
>
> 2009/12/1 Harald Alvestrand <harald at alvestrand.no<mailto:harald at alvestrand.no>>
>>
>> Mark Davis ☕ wrote:
>>>
>>> As far as Harald's back-of-the-envelope calculations go, they present a
>>> very inaccurate picture of the scale. Here are some more exact figures for
>>> that data.
>>>
>>>   1. 819,600,672    = sample size of documents
>>>   2. 5,000    = links with eszed in the sample
>>>   3. 1,000,000,000,000    = total documents in index (2008)
>>>   4. 1,220    = scaling factor (= total docs / sample size)
>>>   5. 6,100,532    = estimated total links with eszed (= scaling *
>>>      sample eszed links)
>>>
>>> Even this has to be taken with a certain grain of salt, since (a) it is
>>> assuming that the sample is representative (although we have reasonable
>>> confidence in that), and (b) it doesn't weight the "importance" of the links
>>> (in terms of the number of times they are followed), and (c) this data was
>>> collected back in Nov 2008, so we've had another year of growth since then.
>>
>> I obviously need a bigger envelope :-) - I didn't think we had one
>> trillion documents in the 2008 index.
>>
>> One missing number: how many links per document?
>>
>> Obviously #eszed links / #documents can't be the basis of the 0.00001%
>> figure that Erik quoted, because 5000/819600672 = 0.00061005%, not 0.00001%,
>> which is a factor of 60 larger, but if we estimate 60 links per document,
>> the 0.00001% fits nicely as the percentage of links that contain eszed.
>>
>>              Harald
>>
>>
>>
>
>
> _______________________________________________
> Idna-update mailing list
> Idna-update at alvestrand.no<mailto:Idna-update at alvestrand.no>
> http://www.alvestrand.no/mailman/listinfo/idna-update
>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.alvestrand.no/pipermail/idna-update/attachments/20091201/d4526ecc/attachment-0001.htm