[RTW] List is now open

[Stefan Håkansson LK]

Another advantage of denoting Stream as something that is uncoded is that it makes it simpler to add audio processing. 

You could have something like

<device> ==> [encoder] -> [transport] -> [decoder] ==> [processing] ==> <audio>

where processing could be something based on for example the rendered scene in a game (e.g. panning).

--Stefan

-----Original Message-----
From: Adam Bergkvist 
Sent: den 25 oktober 2010 14:29
To: Ian Hickson; Justin Uberti
Cc: Stefan Håkansson LK; Silvia Pfeiffer; Harald Alvestrand; rtc-web at alvestrand.no; David Singer
Subject: RE: [RTW] List is now open

On Mon, 11 Oct 2010, Ian Hickson wrote:

> On Sun, 10 Oct 2010, Justin Uberti wrote:
>> 
>> My idea is that we will tie these various pieces together like a 
>> filter chain, i.e.
>> 
>> <device> -> [encoder] -> [transport] -> [decoder] -> <video>
> 
> This is basically what's in the HTML spec today:
> 
> On the sending side:
> 
>   <device> is <device>
>   [encoder] is a Stream object
>   [transport] is a PeerConnection object
> 
> On the receiving side:
> 
>   [transport] is a PeerConnection object
>   [decoder] is a Stream object's .url attribute
>   <video> is a <video>
>

We see Stream simply as the glue between the device element and the transport endpoint, and between the remote transport endpoint and the video element. Thus encoding and decoding is handled by the ConnectionPeer object.

<device> ==> [encoder] -> [transport] -> [decoder] ==> <video>

On the sending side:

  <device>                 is <device>
  ==>                      is a Stream object
  [encoder] -> [transport] is a ConnectionPeer object

On the receiving side:

  [transport] -> [decoder] is a ConnectionPeer object
  ==>                      is a Stream object
  <video>                  is a <video>


In the local self-view case there's no encoding or decoding.

<device> ==> <video>


In the recording case, the encoding is handled by the StreamRecorder.

<device> ==> [recorder] -> [encoder] -> [file]

  <device>                is <device>
  ==>                     is a Stream object
  [recorder] -> [encoder] is a StreamRecorder
  [file]                  is a File

> The details are abstracted out a lot at this point; the idea is that 
> future versions of the API can add specific knobs and dials to allow 
> authors to control what they want to change; we can decide what to 
> expose based on what people most want.
> This dramatically limits the complexity of what we have to implement, 
> and makes it more likely we'll get interop sooner.
> However, I need to do some work on this spec to merge in the ideas 
> from some of Justin's work, so this may change a bit.
> What will certainly change is the way the low-bandwidth channel 
> maintained by the script is interfaced with on the PeerConnection 
> object; I also need to move some of the Session stuff into the spec, 
> e.g. sending of DTMF tones.
>

Rather than specific functionality for sending DTMF tones, it would be more useful to have a generic way to include pre-recorded media in a Stream. 

BR
Adam