<br><br><div class="gmail_quote">On Thu, Oct 13, 2011 at 12:18 AM, Varun Singh <span dir="ltr"><<a href="mailto:vsingh.ietf@gmail.com">vsingh.ietf@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im">On Wed, Oct 12, 2011 at 23:48, Randell Jesup <<a href="mailto:randell-ietf@jesup.org">randell-ietf@jesup.org</a>> wrote:<br>
> On 10/12/2011 11:25 AM, Henrik Lundin wrote:<br>
>><br>
</div><div class="im">> I honestly don't have a lot of love for fully offline, non-real-time<br>
> simulations like ns2 for this sort of work - the realtime algorithms have<br>
> often are painful to rework to run paced by the simulation, and to be honest<br>
> I want a reasonable approximation of a real network, where subjective<br>
> measurements and evaluation can be quickly done. Plus I find Dummynet/Netem<br>
> more intuitive than what I know about using ns2 (which I've never actually<br>
> used, but the comments from Varun Singh make me shudder at using it as a<br>
> primary tool.<br>
><br>
<br>
</div>Just to clarify, in our simulation setup, we hooked up an encoder and<br>
decoder to the NS2 application. If we were simulating dummy packets<br>
then this simulation would run more quickly. The encoder (which runs<br>
outside the NS2 context) provides the H.264 frames to the ns2<br>
application where it is fragmented and encapsulated in RTP packets and<br>
transmitted over the NS2 topology. At the receiver side ns2<br>
application the RTP packet is passed to a real-world decoder. The main<br>
reason for slowness of the simulation is that the ns2 application had<br>
to routinely time sync with the codec (outside the NS2). It was more<br>
like NS2 polling the codec and saying now is 0ms do you have any data,<br>
5ms, 10ms, 15ms, etc.<br>
<br>
The advantage of using NS2 is that you can have complex topologies,<br>
change the type of links (DSL, WLAN, 3G), queues (drop tail, RED),<br>
type of cross-traffic (FTP, CBR, start and stop TCP flows) and easily<br>
measure throughput for each flow (making analysis easier). In our<br>
setup, we also got a YUV file at the decoder with which we could<br>
calculate PSNR and/or play back next to the original YUV.<br></blockquote><div><br></div><div>This is what I am looking for. I was primarily considering using some kind of dummy load generator (video and/or audio) to produce flows with the right characteristics (frame rate, frame size variability, I frames, etc.) and inject this into the simulated network. Looking at metrics such as throughput, end-to-end delay and loss rate would provide a lot of insight into how a real media stream would be affected.</div>
<div><br></div><div>Thanks for the feedback. It seems ns2 is the weapon of choice.</div><div><br></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex;">
<div class="im"><br>
> My preference is for netem and/or dummynet across a bench and also for<br>
> connecting a device to a corporate or high-speed residential network while<br>
> simulating a much lower-speed or loaded access link. Hook up a switch or<br>
> NAT and a PC to run transfers, make VoIP calls, run bittorrent(!!), randomly<br>
> browse, watch Youtube, etc while making video calls on the target. We also<br>
> used to have regular residential cable modems as well. Make lots of actual<br>
> calls where you're talking to someone; humans are very good at picking out<br>
> most types of errors - especially the ones that matter, and you can also<br>
> collect statistics while doing so. Combine those tools with a few good<br>
> testers and you'll find all sorts of funny edge conditions that most<br>
> straight simulations miss.<br>
><br>
<br>
</div>IMO dummynet would work but to properly analyze the rate control, we<br>
must also analyse the end-to-end throughput (and/or other metrics) of<br>
the rtcweb media, link, and other traffic. To make sure that the media<br>
is fair to the cross traffic and that the cross traffic doesn't starve<br>
the media (e.g., in case of Bittorrent or too many TCP). For most<br>
scenarios this should be possible using tcpdump.<br>
<br>
<br>
PlanetLab will be useful if we want to emulate more complex scenarios<br>
(like different types of routers etc). For simple cases, using<br>
dummynet on a single hop (or at end-points) may be good enough.<br>
<div class="HOEnZb"><div class="h5"><br>
> IMHO.<br>
><br>
>> 5. It appears that the code in remote_rate_control.cc<br>
>> (i.e.receiver)<br>
>> currently starts at the max-configured bitrate; is this<br>
>> correct?<br>
>> What's the impact of this; what does the sender-side start at?<br>
>><br>
>><br>
>> This is correct, and means that the receiver side does not<br>
>> enforce or<br>
>> ask for any rate reduction until it has experienced the first<br>
>> over-bandwidth.<br>
>><br>
>><br>
>> Ok, as per above and Magnus's comments, I tend to disagree with this<br>
>> - it almost guarantees you'll be in recovery right at the start of<br>
>> the call, which is not the best experience for 1-to-1 communication,<br>
>> IMHO (and in my experience). See my extensive discussion of<br>
>> starting bandwidths on rtcweb (I should copy it here for archival).<br>
>><br>
>><br>
>> I think there has been a misunderstanding here. It is true that the code<br>
>> in the receiver-side of the CC algorithm (specifically in<br>
>> remote_rate_control.cc) starts at the max-configured rate. This is while<br>
>> waiting for it to detect the first over-use. However, and this is were<br>
>> the catch is, it does not mean that the sender starts sending at the max<br>
>> rate. In our current implementation, the sender decides the actual start<br>
>> rate. Further, since the original implementation was done based on<br>
>> TMMBR, we had to implement a stand-alone CC at the sender too, since<br>
>> it's not allowed to rely on TMMBR alone. Thus, as long as the TMMBR<br>
>> feedback is larger than what the sender's internal CC says, it will not<br>
>> listen to the TMMBR. Therefore, the receive-side initialization that you<br>
>> see does not mean that we start each call at ludicrous speed. Some of<br>
>> the things in the implementation are legacy...<br>
><br>
> Ok, good - I thought I'd asked that question when we all were chatting and I<br>
> probably wasn't specific enough.<br>
><br>
>> Hmmm. Oh well. I guarantee there are devices out there that drift<br>
>> a lot... And even have time go backwards (great fun there).<br>
>><br>
>> I'm sure there are. So far we have not seen drift large enough to<br>
>> actually offset the over-/under-use detection.<br>
><br>
> I'm told some PC audio cards/interfaces have significantly drifty timebases<br>
> - but perhaps not at a level that matters here. When working with the old<br>
> GIPS code, we kept a PLL tracking the apparent timebase of each far-end<br>
> stream, updated on RTCP reports, and reset if something "odd" happens.<br>
><br>
><br>
> --<br>
> Randell Jesup<br>
> <a href="mailto:randell-ietf@jesup.org">randell-ietf@jesup.org</a><br>
> _______________________________________________<br>
> Rtp-congestion mailing list<br>
> <a href="mailto:Rtp-congestion@alvestrand.no">Rtp-congestion@alvestrand.no</a><br>
> <a href="http://www.alvestrand.no/mailman/listinfo/rtp-congestion" target="_blank">http://www.alvestrand.no/mailman/listinfo/rtp-congestion</a><br>
><br>
<br>
<br>
<br>
</div></div><span class="HOEnZb"><font color="#888888">--<br>
<a href="http://www.netlab.tkk.fi/~varun/" target="_blank">http://www.netlab.tkk.fi/~varun/</a><br>
</font></span><div class="HOEnZb"><div class="h5">_______________________________________________<br>
Rtp-congestion mailing list<br>
<a href="mailto:Rtp-congestion@alvestrand.no">Rtp-congestion@alvestrand.no</a><br>
<a href="http://www.alvestrand.no/mailman/listinfo/rtp-congestion" target="_blank">http://www.alvestrand.no/mailman/listinfo/rtp-congestion</a><br>
</div></div></blockquote></div><br>