Implementation questions
Harald Alvestrand
harald at alvestrand.no
Mon Dec 22 12:15:28 CET 2008
Mark Davis wrote:
>
>
> *IDNA Implementation Questions*
>
> *
> *In looking at how to implement 2008 (and maintain backward
> compatibility with 2003), we are wrestling with some practical
> questions that we'd appreciate feedback on. These are not questioning
> the spec, but rather trying to see how to apply it to some practical
> cases we've come up against.
>
>
> Scenario
>
> Look at the following scenario, where we have three processes that
> handle an IRI (perhaps just passing it through), with the final one
> using it to access the DNS. (We'll use the term IRI for both when the
> domain name labels are in punycode or in Unicode. They aren't
> necessarily known to be A-Labels or U-Labels at any given point.)
>
> P1 => P2 => P3 => P4 => DNS
This is a very open-ended configuration. But let's shoot.
>
>
> Questions
>
>
> 1. Suppose that P2 is on Unicode 5.1, and the others are on Unicode
> 6.0. If P2 does a validity check, then it could prevent a perfectly
> valid IRI from being correctly looked up. To prevent this problem,
> does that mean that the best practice is for only P4 to do validity
> checking? Or should the others do some weaker form of validity
> checking, like skipping a check for UNASSIGNED?
If P2 checks at all, P2 should check for UNASSIGNED.
If P2 trusts P1 so much that it can skip the check for UNASSIGNED, it
should trust P1 enough to skip all checks, which violates the scenario
restriction above.
There is no "perfectly valid" IRI; there are IRIs that are valid under
Unicode 6.0, and there are IRIs that are valid under Unicode 5.1. For
that matter, there are IRIs that are valid under Unicode 7.0. We have
accepted that these can't be looked up until upgrades happen.
This is exactly the same class of processing logic needed if P1->Pn are
expecting NFC-compliant UTF-8. Either they trust that input is valid, or
they check.
>
> 2. Suppose P3 is a non-IDNA aware process, so IRIs should be converted
> to Punycode by P2 before sending. Should one do a validity check in
> P2? How do we avoid problem #1 in that case?
Given the answer to #1, this is not an issue. P2 should check.
>
> 3. The current protocol spec appears to only require validity checking
> when converting to punycode. So when an IRI is already in punycode
> (which could have been from IDNA2003 application), it might not
> undergo any checking at all when going from P1 to the DNS; so
> everything depends on the registry's doing the right thing. Is it best
> to check anyway, or does that run into problem #1?
See above about "trust". Either P2.... trust P1 to have done the checks,
or they don't. If they do checks at all, they need to do all checks.
>
> 4. If P2 accepts an IRI in Unicode and passes it on to P3 in Unicode
> (never converting to punycode), should it do any validity checking?
See above.
>
> 5. When a search engine does indexing, it has to map together IRIs
> that are "equivalent" (resolving to the same logical location). When
> it provides an IRI to the user for a page, that IRI should go to the
> indexed page. However, because IDNA2003 and IDNA2008 browsers may go
> to different places with the same IRI, which do we provide? If we try
> to test for which browser the user has, that is clumsy and error-prone.
When a search engine does indexing, it maps together a lot more URLs
than the ones that appear superficially syntactically equivalent.
My personal answer is that the URI (with a punycoded domain name) should
be provided, because that's what the search engine has actually observed
to cause the page to be fetched), and reconstructing an IRI from an URI
is an error prone process (you basically have to guess).
Given that some people have decided that they want to provide IRIs to
the user, some A-labels have to be converted back to U-labels. In both
the cases we have let ourselves be twisted around by (final sigma and
ß), the punycode form maps back to exactly one U-label.
This U-label will cause a different A-label in an IDNA2003 browser than
in an IDNA2008 browser. But there's absolutely no ambiguity on which
U-label to return; there is no possible U-label that one can return to
an IDNA2003 browser that can cause that browser to go to the IDNA2008 site.
So the answer is simple: Return the A-label form. Preferably by
abandoning the idea of returning IRIs.
Harald
More information about the Idna-update
mailing list