[R-C] Congestion Control BOF

[Varun Singh]

Hi Henrik,

a comment inline.

On Thu, Oct 13, 2011 at 09:37, Henrik Lundin <henrik.lundin at webrtc.org> wrote:
>
>
> On Thu, Oct 13, 2011 at 12:18 AM, Varun Singh <vsingh.ietf at gmail.com> wrote:
>>
>> On Wed, Oct 12, 2011 at 23:48, Randell Jesup <randell-ietf at jesup.org>
>> wrote:
>> > On 10/12/2011 11:25 AM, Henrik Lundin wrote:
>> >>
>> > I honestly don't have a lot of love for fully offline, non-real-time
>> > simulations like ns2 for this sort of work - the realtime algorithms
>> > have
>> > often are painful to rework to run paced by the simulation, and to be
>> > honest
>> > I want a reasonable approximation of a real network, where subjective
>> > measurements and evaluation can be quickly done.  Plus I find
>> > Dummynet/Netem
>> > more intuitive than what I know about using ns2 (which I've never
>> > actually
>> > used, but the comments from Varun Singh make me shudder at using it as a
>> > primary tool.
>> >
>>
>> Just to clarify, in our simulation setup, we hooked up an encoder and
>> decoder to the NS2 application. If we were simulating dummy packets
>> then this simulation would run more quickly. The encoder (which runs
>> outside the NS2 context) provides the H.264 frames to the ns2
>> application where it is fragmented and encapsulated in RTP packets and
>> transmitted over the NS2 topology. At the receiver side ns2
>> application the RTP packet is passed to a real-world decoder. The main
>> reason for slowness of the simulation is that the ns2 application had
>> to routinely time sync with the codec (outside the NS2). It was more
>> like NS2 polling the codec and saying now is 0ms do you have any data,
>> 5ms, 10ms, 15ms, etc.
>>
>> The advantage of using NS2 is that you can have complex topologies,
>> change the type of links (DSL, WLAN, 3G), queues (drop tail, RED),
>> type of cross-traffic (FTP, CBR, start and stop TCP flows) and easily
>> measure throughput for each flow (making analysis easier). In our
>> setup, we also got a YUV file at the decoder with which we could
>> calculate PSNR and/or play back next to the original YUV.
>
> This is what I am looking for. I was primarily considering using some kind
> of dummy load generator (video and/or audio) to produce flows with the right
> characteristics (frame rate, frame size variability, I frames, etc.) and
> inject this into the simulated network. Looking at metrics such as
> throughput, end-to-end delay and loss rate would provide a lot of insight
> into how a real media stream would be affected.
> Thanks for the feedback. It seems ns2 is the weapon of choice.


We wrote a very simple packet generator that simulates I and P packets
(no B frames). It uses a customizable but fixed GOP size and varying
bit rate. IMO this doesn't work well for complex/stress testing,
because in the real-world the codecs don't work in such a
deterministic way. A lot also depends on the type of content.

I agree with Randell Jesup that with a real world implementation it
easier to make out video artifacts etc. Our recent experiments with
Dummynet and NS2 show that there is some mismatch between the
simulations. Dummynet simulations are turning out to be worse off. I
am not ruling out that we may have bugs in our simulation setup/RTP
code (our NS2 code is more mature) but simple things like RTT
variation in case of dummynet is much more than the modelled one in
NS2 for constant capacity and CBR media. Which I think may be also
true for real-world deployments.

>>
>> > My preference is for netem and/or dummynet across a bench and also for
>> > connecting a device to a corporate or high-speed residential network
>> > while
>> > simulating a much lower-speed or loaded access link.  Hook up a switch
>> > or
>> > NAT and a PC to run transfers, make VoIP calls, run bittorrent(!!),
>> > randomly
>> > browse, watch Youtube, etc while making video calls on the target.  We
>> > also
>> > used to have regular residential cable modems as well.  Make lots of
>> > actual
>> > calls where you're talking to someone; humans are very good at picking
>> > out
>> > most types of errors - especially the ones that matter, and you can also
>> > collect statistics while doing so.  Combine those tools with a few good
>> > testers and you'll find all sorts of funny edge conditions that most
>> > straight simulations miss.
>> >
>>
>> IMO dummynet would work but to properly analyze the rate control, we
>> must also analyse the end-to-end throughput (and/or other metrics) of
>> the rtcweb media, link, and other traffic. To make sure that the media
>> is fair to the cross traffic and that the cross traffic doesn't starve
>> the media (e.g., in case of Bittorrent or too many TCP). For most
>> scenarios this should be possible using tcpdump.
>>
>>
>> PlanetLab will be useful if we want to emulate more complex scenarios
>> (like different types of routers etc). For simple cases, using
>> dummynet on a single hop (or at end-points) may be good enough.
>>
>> > IMHO.
>> >
>> >>            5. It appears that the code in remote_rate_control.cc
>> >>        (i.e.receiver)
>> >>            currently starts at the max-configured bitrate; is this
>> >> correct?
>> >>            What's the impact of this; what does the sender-side start
>> >> at?
>> >>
>> >>
>> >>        This is correct, and means that the receiver side does not
>> >>        enforce or
>> >>        ask for any rate reduction until it has experienced the first
>> >>        over-bandwidth.
>> >>
>> >>
>> >>    Ok, as per above and Magnus's comments, I tend to disagree with this
>> >>    - it almost guarantees you'll be in recovery right at the start of
>> >>    the call, which is not the best experience for 1-to-1 communication,
>> >>    IMHO (and in my experience).  See my extensive discussion of
>> >>    starting bandwidths on rtcweb (I should copy it here for archival).
>> >>
>> >>
>> >> I think there has been a misunderstanding here. It is true that the
>> >> code
>> >> in the receiver-side of the CC algorithm (specifically in
>> >> remote_rate_control.cc) starts at the max-configured rate. This is
>> >> while
>> >> waiting for it to detect the first over-use. However, and this is were
>> >> the catch is, it does not mean that the sender starts sending at the
>> >> max
>> >> rate. In our current implementation, the sender decides the actual
>> >> start
>> >> rate. Further, since the original implementation was done based on
>> >> TMMBR, we had to implement a stand-alone CC at the sender too, since
>> >> it's not allowed to rely on TMMBR alone. Thus, as long as the TMMBR
>> >> feedback is larger than what the sender's internal CC says, it will not
>> >> listen to the TMMBR. Therefore, the receive-side initialization that
>> >> you
>> >> see does not mean that we start each call at ludicrous speed. Some of
>> >> the things in the implementation are legacy...
>> >
>> > Ok, good - I thought I'd asked that question when we all were chatting
>> > and I
>> > probably wasn't specific enough.
>> >
>> >>    Hmmm.  Oh well.  I guarantee there are devices out there that drift
>> >>    a lot...  And even have time go backwards (great fun there).
>> >>
>> >> I'm sure there are. So far we have not seen drift large enough to
>> >> actually offset the over-/under-use detection.
>> >
>> > I'm told some PC audio cards/interfaces have significantly drifty
>> > timebases
>> > - but perhaps not at a level that matters here.  When working with the
>> > old
>> > GIPS code, we kept a PLL tracking the apparent timebase of each far-end
>> > stream, updated on RTCP reports, and reset if something "odd" happens.
>> >
>> >
>> > --
>> > Randell Jesup
>> > randell-ietf at jesup.org
>> > _______________________________________________
>> > Rtp-congestion mailing list
>> > Rtp-congestion at alvestrand.no
>> > http://www.alvestrand.no/mailman/listinfo/rtp-congestion
>> >
>>
>>




-- 
http://www.netlab.tkk.fi/~varun/