[RTW] Requirement F9 - Audio processing

[Kavan Seggie]

Hi all

Firstly congratulations on a great initiative. Video and voice chat in the
web browser is incredibly powerful and will without doubt change the
communications industry.

A quick introduction. We are a small startup that over a two year period
have created a video chat plugin. Due to our limited resources we have used
open source technologies where ever possible. The one area that is very
difficult to find quality open source libraries is in audio processing.

As you all probably know, the overall communications experience of a video
chat is greatly dependent on audio quality.  The audio codec is important,
but just as important are the other DSP elements like the jitter buffer,
AGC, noise suppression and AEC. Without this audio processing even the best
video and audio can be useless. This is what happened with the Flash plugin
and AEC became the most/2nd most requested feature in Flash. They have
finally implemented AEC, AGC and noise suppression in 10.3 beta and it is
now usable as a communications tool. Before, without the audio processing,
 it wasn't in my opinion.

To the best of my knowledge the only reasonable open sourced software in
this field is Jean-Marc Valin's Speex DSP library. We have been working on
optimising this for a few months and will continue to do so. But it is a
basic implementation and to be honest our optimisations are not great yet.
The infinite permutations of hardware, differing latency in the network and
the system, and the different environments the user can be in, make this an
incredibly difficult problem to solve. Also if we want to support different
audio codecs, these libraries need to be optimised for each codec.

I have read in the proposed documents that AEC is a requirement (F9) but I
have not been able to find any plans on addressing the issue.

After Google bought GIPS, I thought that Google may open source GIPS's rock
solid audio technology. However that was almost a year ago so there may be
issues with this. It is also just a guess, Google may have other plans for
the technology.

So perhaps, since this is an important requirement for the adoption of RTC,
the group could work on bettering the Speex algorithms? We would be happy
to contribute our amendments to the Speex algorithms as well as other work
we have done, and so kick start a project. We could also commit our audio
engineer to work on it almost full time. He has considerable experience in
creating voice codecs and is learning his way through the other DSP
elements.

All thoughts and recommendations welcome.

Regards


Kavan Seggie
SayMama
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://www.alvestrand.no/pipermail/rtc-web/attachments/20110315/6b2c7c45/attachment.html>