[R-C] Strawman: Requirements

[Harald Alvestrand]

On 10/19/2011 03:37 AM, Wesley Eddy wrote:
> On 10/18/2011 10:07 AM, Harald Alvestrand wrote:
>>
> > ...
>>
>> What do you think?
>>
>
> A good start; thanks for kicking off this part of the discussion.
>
> I think it would be useful to structure tests around the expectations
> that BCP 133 / RFC 5033 describe.
Good point.
We're in the situation where we have certain functionality we wish to 
deploy on the Internet.
RFC 5033 is written in the form of "you need to evaluate", but 
completely omits the pass/fail criteria.
I think we're in a position to say that "if we can't deploy our stuff, 
the algorithm's a failure".

>   Some of the things in the "required
> results" list you sent speak towards things in that RFC, but not fully,
> and the approach doesn't seem quite right; for instance, why is 30%
> a magic number?
Because a number is easier to test for than "reasonably".
Perfection would, in many cases, be 50%, but perfection isn't necessary 
for deployment.
RFC 5033 contains statements like this:

    (5) Fairness within the Alternate Congestion Control Algorithm.

        In environments with multiple competing flows all using the same
        alternate congestion control algorithm, the proposal should
        explore how bandwidth is shared among the competing flows.

It doesn't say anything about what's acceptable or unacceptable with 
regard to how bandwidth is shared; it's impossible to construct a test 
for verifying whether what we have is "good enough" based on this criterion.
So we're on our own when it comes to making these things concrete.
>   RFC 5166 provides metrics that we will probably want
> to capture an analyze.
Definitely, as a part of experimentation. For criteria writing, I want 
to look for cases where a particular measurement of a particular metric 
would cause us to say "not good enough".
> Another document that is more geared towards TCP
> simulations, but may be helpful for ideas, is draft-irtf-tmrg-tests.
Reading!
>
>