Re: [pjsip] Audio latency in Windows Mobile

NI
Nanang Izzuddin
Tue, Dec 15, 2009 9:42 AM

Hi,

Looking at this JB stats:

JB summary:
....
burst (min/max/avg/dev)=2/98/33/29 frames

Such high-burst would introduce high-latency indeed, so we could
easily guess that the 3xx ms latency was spent in JB. So, the problem
now should be where the burst came from and also perhaps, why.

Frankly speaking, burst calculated from the number of consecutive
frames received from network (PUT operation to JB) between two
adjacent frame deliveries to audio device (GET operation from JB
operation).

Now, let's see this simplified audio delivery path:

Peer mic -> network -> JB -> local speaker.

All elements, except JB, potentially introduce burst. And using some
tools we could detect burst level of each element, for example:

  • use pjsystest, to check burst level of both peer mic and local speaker.
  • if the accumulation of both burst levels is much lower than 380ms,
    then we could say that it could be the network, however there may
    still be other factor, see note below.
  • you may also use network tool, e.g: ping, to check network burst and
    latency (note that different bandwidth may result different burst
    level/latency).

Note that the pjsystest runs audio device test on 'idle' CPU, while a
call session involves many other processings, e.g: audio processing
(codec, resample, EC, etc), logging, UI, which may introduce CPU
spikes and CPU spikes could disturb audio device clock (and perhaps
also introduce network burst issue?). Where in the end, it leads to
audio latency and/or impairment.

So, if you found the audio devices tests are fine and the network is
also known to be fine, perhaps you need to do some optimizations,
check the FAQ [1] :) And please share with us whatever you find and
achieve :)


[1] http://trac.pjsip.org/repos/wiki/FAQ#cpu

BR,
nanang

On Fri, Dec 11, 2009 at 11:10 PM, Nuno Costa
nuno.costa@wit-software.com wrote:

Hi Nanang,

Thank you for your reply.

I know the audio device is introducing some latency (around 170ms, measured
by pjsystest).
But I have a network latency of 100ms (max) and an end-to-end latency of
650ms.

This means that there are 380ms being spent somewhere else, right?
From the logs, it seems to me that might be during network processor because
there are a lot of warnings on the jitter buffer becoming empty.

Am I completely wrong?

Note: I'm using G.711 on all my tests.

Best regards,
Nuno Costa

Hi, Looking at this JB stats: --- JB summary: .... burst (min/max/avg/dev)=2/98/33/29 frames --- Such high-burst would introduce high-latency indeed, so we could easily guess that the 3xx ms latency was spent in JB. So, the problem now should be where the burst came from and also perhaps, why. Frankly speaking, burst calculated from the number of consecutive frames received from network (PUT operation to JB) between two adjacent frame deliveries to audio device (GET operation from JB operation). Now, let's see this simplified audio delivery path: --- Peer mic -> network -> JB -> local speaker. --- All elements, except JB, potentially introduce burst. And using some tools we could detect burst level of each element, for example: - use pjsystest, to check burst level of both peer mic and local speaker. - if the accumulation of both burst levels is much lower than 380ms, then we could say that it could be the network, however there may still be other factor, see note below. - you may also use network tool, e.g: ping, to check network burst and latency (note that different bandwidth may result different burst level/latency). Note that the pjsystest runs audio device test on 'idle' CPU, while a call session involves many other processings, e.g: audio processing (codec, resample, EC, etc), logging, UI, which may introduce CPU spikes and CPU spikes could disturb audio device clock (and perhaps also introduce network burst issue?). Where in the end, it leads to audio latency and/or impairment. So, if you found the audio devices tests are fine and the network is also known to be fine, perhaps you need to do some optimizations, check the FAQ [1] :) And please share with us whatever you find and achieve :) --- [1] http://trac.pjsip.org/repos/wiki/FAQ#cpu --- BR, nanang On Fri, Dec 11, 2009 at 11:10 PM, Nuno Costa <nuno.costa@wit-software.com> wrote: > Hi Nanang, > > Thank you for your reply. > > I know the audio device is introducing some latency (around 170ms, measured > by pjsystest). > But I have a network latency of 100ms (max) and an end-to-end latency of > 650ms. > > This means that there are 380ms being spent somewhere else, right? > From the logs, it seems to me that might be during network processor because > there are a lot of warnings on the jitter buffer becoming empty. > > Am I completely wrong? > > Note: I'm using G.711 on all my tests. > > Best regards, > Nuno Costa > >
NI
Nanang Izzuddin
Tue, Dec 15, 2009 10:02 AM

Forgot to mention one thing.

Audio device clock drift may also cause latency, e.g: peer mic
produces frames faster than local speaker can consume, as after
sometime, this tends to make JB full. JB actually has got capability
to detect drift and reduce latency caused by drift, however, it is not
so aggressive (to reserve audio quality), so latency may be decreased
rather slowly.

However, it seems that the main problem in your case is burst.

BR,
nanang

On Tue, Dec 15, 2009 at 4:42 PM, Nanang Izzuddin nanang@pjsip.org wrote:

Hi,

Looking at this JB stats:

JB summary:
   ....
   burst (min/max/avg/dev)=2/98/33/29 frames

Such high-burst would introduce high-latency indeed, so we could
easily guess that the 3xx ms latency was spent in JB. So, the problem
now should be where the burst came from and also perhaps, why.

Frankly speaking, burst calculated from the number of consecutive
frames received from network (PUT operation to JB) between two
adjacent frame deliveries to audio device (GET operation from JB
operation).

Now, let's see this simplified audio delivery path:

Peer mic -> network -> JB -> local speaker.

All elements, except JB, potentially introduce burst. And using some
tools we could detect burst level of each element, for example:

  • use pjsystest, to check burst level of both peer mic and local speaker.
  • if the accumulation of both burst levels is much lower than 380ms,
    then we could say that it could be the network, however there may
    still be other factor, see note below.
  • you may also use network tool, e.g: ping, to check network burst and
    latency (note that different bandwidth may result different burst
    level/latency).

Note that the pjsystest runs audio device test on 'idle' CPU, while a
call session involves many other processings, e.g: audio processing
(codec, resample, EC, etc), logging, UI, which may introduce CPU
spikes and CPU spikes could disturb audio device clock (and perhaps
also introduce network burst issue?). Where in the end, it leads to
audio latency and/or impairment.

So, if you found the audio devices tests are fine and the network is
also known to be fine, perhaps you need to do some optimizations,
check the FAQ [1] :) And please share with us whatever you find and
achieve :)


[1] http://trac.pjsip.org/repos/wiki/FAQ#cpu

BR,
nanang

On Fri, Dec 11, 2009 at 11:10 PM, Nuno Costa
nuno.costa@wit-software.com wrote:

Hi Nanang,

Thank you for your reply.

I know the audio device is introducing some latency (around 170ms, measured
by pjsystest).
But I have a network latency of 100ms (max) and an end-to-end latency of
650ms.

This means that there are 380ms being spent somewhere else, right?
From the logs, it seems to me that might be during network processor because
there are a lot of warnings on the jitter buffer becoming empty.

Am I completely wrong?

Note: I'm using G.711 on all my tests.

Best regards,
Nuno Costa

Forgot to mention one thing. Audio device clock drift may also cause latency, e.g: peer mic produces frames faster than local speaker can consume, as after sometime, this tends to make JB full. JB actually has got capability to detect drift and reduce latency caused by drift, however, it is not so aggressive (to reserve audio quality), so latency may be decreased rather slowly. However, it seems that the main problem in your case is burst. BR, nanang On Tue, Dec 15, 2009 at 4:42 PM, Nanang Izzuddin <nanang@pjsip.org> wrote: > Hi, > > Looking at this JB stats: > --- > JB summary: >    .... >    burst (min/max/avg/dev)=2/98/33/29 frames > --- > Such high-burst would introduce high-latency indeed, so we could > easily guess that the 3xx ms latency was spent in JB. So, the problem > now should be where the burst came from and also perhaps, why. > > Frankly speaking, burst calculated from the number of consecutive > frames received from network (PUT operation to JB) between two > adjacent frame deliveries to audio device (GET operation from JB > operation). > > Now, let's see this simplified audio delivery path: > --- > Peer mic -> network -> JB -> local speaker. > --- > All elements, except JB, potentially introduce burst. And using some > tools we could detect burst level of each element, for example: > - use pjsystest, to check burst level of both peer mic and local speaker. > - if the accumulation of both burst levels is much lower than 380ms, > then we could say that it could be the network, however there may > still be other factor, see note below. > - you may also use network tool, e.g: ping, to check network burst and > latency (note that different bandwidth may result different burst > level/latency). > > Note that the pjsystest runs audio device test on 'idle' CPU, while a > call session involves many other processings, e.g: audio processing > (codec, resample, EC, etc), logging, UI, which may introduce CPU > spikes and CPU spikes could disturb audio device clock (and perhaps > also introduce network burst issue?). Where in the end, it leads to > audio latency and/or impairment. > > So, if you found the audio devices tests are fine and the network is > also known to be fine, perhaps you need to do some optimizations, > check the FAQ [1] :) And please share with us whatever you find and > achieve :) > > --- > [1] http://trac.pjsip.org/repos/wiki/FAQ#cpu > --- > > BR, > nanang > > > > On Fri, Dec 11, 2009 at 11:10 PM, Nuno Costa > <nuno.costa@wit-software.com> wrote: >> Hi Nanang, >> >> Thank you for your reply. >> >> I know the audio device is introducing some latency (around 170ms, measured >> by pjsystest). >> But I have a network latency of 100ms (max) and an end-to-end latency of >> 650ms. >> >> This means that there are 380ms being spent somewhere else, right? >> From the logs, it seems to me that might be during network processor because >> there are a lot of warnings on the jitter buffer becoming empty. >> >> Am I completely wrong? >> >> Note: I'm using G.711 on all my tests. >> >> Best regards, >> Nuno Costa >> >> >