interoperability testing

[Erik van der Poel]

Hi Eric,

I'm not really sure how to respond, but we generally don't mention
APIs much in our IDNA "interoperability story". Until now, IDNA2003
has been quite clear, and so the APIs all implemented the same thing
(more or less), e.g. Microsoft's API(s), GNU's libidn, etc. With
IDNAbis, we may actually see two different kinds of APIs, one that is
more or less the same as IDNA2003 (with mappings) and one that does
not perform mappings, especially if the email specs specify U-labels
*and* the major vendors follow suit.

Google Web Search's IDNA "interoperability story" would not be
centered on APIs, but on "wire formats" (for lack of a better term in
this wireless age). Specifically, we are extremely interested in the
HTML, HTTP and DNS wire formats. We study the browsers very closely,
and we mimic the major implementations unless there is a very good
reason to diverge. I should note that Google Web Search has two
"ends": the crawler end, which is as lenient as the most lenient major
implementations (including Opera), and the serving end, which is as
strict as we need to be for the browsers that are still being used
(i.e. we serve Punycode because MSIE6 does not support IDNA).

The story for Chrome is probably quite similar (i.e. they look at the
major browsers). I'm less familiar with Google's email apps and IM
apps.

Does this help at all?

Erik

On Sat, Jul 11, 2009 at 3:24 PM, Eric
Brunner-Williams<ebw at abenaki.wabanaki.net> wrote:
> Erik,
>
> Thanks for the comment. I actually ment "google" when I typed "search
> engine", then backspaced and wrote "database".
>
> The example, path names in a URL, via packet capture, would interest me if
> it were 1989, and I was working on i18n in the *nix file system. I
> appreciate that you've offered this only as an example of what you are doing
> with packet capture, using identical test sets and different instances of
> the same (to a major release no.) browser, and an IDNA proximal issue, but
> I'm still wondering what we really mean by interoperability.
>
> For a universe of values, two actually, the 2003 and the bis universes,
> there is an interoperability "story" for resolution, not a lot different
> from put $vendor_router in $test_harness, load with $config_files and
> $in_packet_streams and log $internal_state and $out_packet_streams. The
> $edit_zonefile issues are similar to the router $config_files tests.
>
> If you, or Shawn, or Gerv, or Pete, could state what the interoperability
> story is for your apps, something that doesn't end up with "and then there
> was phishing, badness and thrashing", it would help me at least.
>
> I'd be happy with "this defines source code portability for some i18n
> related APIs for third-party developers for (I know this is a stretch, but I
> did a lot of these) MUMBLEIX conformant applications." Then the
> interoperability at the API level would be a little clearer.
>
> Eric
>
> Erik van der Poel wrote:
>>
>> At Google, we have been working on a tool and set of test cases in
>> HTML that allow us to test the browsers. (We haven't spent time on IM,
>> email, databases, etc yet.) We don't use CSS in these test cases, so
>> the results don't show any CSS-dependent differences. Basically, we
>> start WireShark (previously known as Ethereal) and then we capture
>> packets sent from browsers when we load the HTML test files in them.
>> Then we save the captured file and generate reports that show
>> differences between, say, one browser on different operating systems,
>> or different browsers on the same operating system.
>>
>> For example, we can now clearly see the difference between Firefox 2
>> and 3 in the handling of non-ASCII text in the path part of the URL.
>> (Firefox 2 sends it in the original encoding of the HTML, while
>> Firefox 3 sends it in UTF-8.) We also test every ASCII value (0..127)
>> in every part of the URL. We look at both DNS and HTTP packets.
>>
>> The results are not quite ready for publication. We will probably send
>> them to the browser developers first, and then to spec writers and
>> other interested folks.
>>
>> Erik
>>
>> On Sat, Jul 11, 2009 at 10:32 AM, Eric
>> Brunner-Williams<ebw at abenaki.wabanaki.net> wrote:
>>
>>>
>>> I'm not sure there is a sensible single definition of IDNAbis
>>> interoperation by multiple implementations. I'm interested if one exists
>>> for IDNA2003.
>>>
>>> If one were to automate a test for two or more instances of a browser,
>>> possibly different snapshots of the same implementation, having common
>>> specification, variation in css handling between the two or more
>>> versions could result in "difference determined by test".
>>>
>>> We are fairly safe when we ask if the resolution narrative is
>>> indifferent to which of {bind8, bind9, ...} instances of and test
>>> configurations of caching and authoritative resolvers.
>>>
>>> We are also fairly safe when we ask if the call and return narrative of
>>> API use is portable within some framework of source code portability, if
>>> APIs are within the scope of interoperability and multiple independent
>>> implementations.
>>>
>>> How do the {IM,email,browser,database,...} application implementors
>>> propose to define meaningful functional interoperation between any two
>>> or more independent implementations of their applications, or across any
>>> test configuration of similar or dissimilar applications?
>>>
>>> Eric
>>>
>>
>>
>>
>
>