Empathy List Archives

MM

Massimiliano Montevecchi

Mon, Sep 22, 2008 1:15 PM

Hi all,

I am evaluating the quality of the speech during a VoIP call between two
PJSUA applications running on two different windows XP hosts.

For this purpose I managed PJSUA with a script so I was able to perform
automatically about 1500 calls.

During a call each peer entity plays a reference speech sample that is
recorded on the other side of the call. Then a speech quality evaluation is
performed using a PESQ tool.

The two machines hosting pjsua application are directly connected trough a
Ethernet switch in order to minimize the network impairment.

The codec used during the call is G.711 and the expected PESQ score for such
codec is 4.40.

The results of my tests highlight the speech quality is instable. That is,
often the PESQ score is the expected one but sometimes (about 10% of total
measures) the score is significatively less (3.50).

Did anyone perform such type of tests or have experience of such type of
speech quality problems?

Best Regards

Massimiliano Montevecchi

Hi all, I am evaluating the quality of the speech during a VoIP call between two PJSUA applications running on two different windows XP hosts. For this purpose I managed PJSUA with a script so I was able to perform automatically about 1500 calls. During a call each peer entity plays a reference speech sample that is recorded on the other side of the call. Then a speech quality evaluation is performed using a PESQ tool. The two machines hosting pjsua application are directly connected trough a Ethernet switch in order to minimize the network impairment. The codec used during the call is G.711 and the expected PESQ score for such codec is 4.40. The results of my tests highlight the speech quality is instable. That is, often the PESQ score is the expected one but sometimes (about 10% of total measures) the score is significatively less (3.50). Did anyone perform such type of tests or have experience of such type of speech quality problems? Best Regards Massimiliano Montevecchi

IK

Ilya Kalinin

Mon, Sep 22, 2008 2:39 PM

Hi.

In my code I am tying to reinitialize media transport before each call
session in order to increment rtp port, I do something like this:

pjsua_transport_config transport_cfg;

pjsua_transport_config_default(&transport_cfg);

pjsua_media_transports_create(&transport_cfg);

after few calls application is crushes on this code segment:

pjsua_media.c

/* Delete existing media transports */

for (i=0; i<pjsua_var.ua_cfg.max_calls; ++i) {

  if (pjsua_var.calls[i].med_tp != NULL) {

      pjmedia_transport_close(pjsua_var.calls[i].med_tp);

      pjsua_var.calls[i].med_tp = NULL;

  }

}

I put this code in pjsua sample and I get same behavior. I'm using now
pjproject 1.0-rc2, with pjproject 0.8 it worked fine.

Do I do something wrong or there is a bug in pjproject?

Thanks in advance.

Hi. In my code I am tying to reinitialize media transport before each call session in order to increment rtp port, I do something like this: pjsua_transport_config transport_cfg; pjsua_transport_config_default(&transport_cfg); pjsua_media_transports_create(&transport_cfg); after few calls application is crushes on this code segment: pjsua_media.c /* Delete existing media transports */ for (i=0; i<pjsua_var.ua_cfg.max_calls; ++i) { if (pjsua_var.calls[i].med_tp != NULL) { pjmedia_transport_close(pjsua_var.calls[i].med_tp); pjsua_var.calls[i].med_tp = NULL; } } I put this code in pjsua sample and I get same behavior. I'm using now pjproject 1.0-rc2, with pjproject 0.8 it worked fine. Do I do something wrong or there is a bug in pjproject? Thanks in advance.

BP

Benny Prijono

Mon, Sep 22, 2008 2:56 PM

On Mon, Sep 22, 2008 at 2:15 PM, Massimiliano Montevecchi <
massimiliano.montevecchi@elsagdatamat.com> wrote:

Hi all,

I am evaluating the quality of the speech during a VoIP call between two
PJSUA applications running on two different windows XP hosts.

For this purpose I managed PJSUA with a script so I was able to perform
automatically about 1500 calls.

During a call each peer entity plays a reference speech sample that is
recorded on the other side of the call. Then a speech quality evaluation is
performed using a PESQ tool.

The two machines hosting pjsua application are directly connected trough a
Ethernet switch in order to minimize the network impairment.

The codec used during the call is G.711 and the expected PESQ score for
such codec is 4.40.

The results of my tests highlight the speech quality is instable. That is,
often the PESQ score is the expected one but sometimes (about 10% of total
measures) the score is significatively less (3.50).

Did anyone perform such type of tests or have experience of such type of
speech quality problems?

Thanks for doing the tests and sharing the results. We also have PESQ tests
as part of the automated unit tests framework (Python based, on
pjsip-apps/src/test-pjsua directory), and got the report about intermittent
audio degradation too.

Our suspicion now lies with the jitter buffer. During the initial call
establishment, perhaps due to high activity in the signaling thread, or the
difference in the call establishment time between caller an callee, some RTP
packets will be queued in the socket buffer and once the media is started
these RTP packets will be stored in the jbuf in a burst.

Often this burst exceeds the jbuf maximum size (it was 340ms), hence it will
be discarded. At other times, some frames will also be discarded by the jbuf
when it tries to optimize the latency. These discard operations will cause
click noise in the playback, causing the PESQ score to degrade.

That's probably the cause of your results.

We have a ticket for this (http://trac.pjsip.org/repos/ticket/638) and
Nanang has a pending commit for this ticket, hopefully the situation will
improve by then. It will be good if you could retest again with the new
changes then, to get a second opinion on this.

How long did you set the call duration to? I suspect we will get better PESQ
score if you run the call to longer duration, after the media is stabilized
after the initial setup activity.

Cheers
Benny

Best Regards

Massimiliano Montevecchi

On Mon, Sep 22, 2008 at 2:15 PM, Massimiliano Montevecchi < massimiliano.montevecchi@elsagdatamat.com> wrote: > Hi all, > > I am evaluating the quality of the speech during a VoIP call between two > PJSUA applications running on two different windows XP hosts. > > For this purpose I managed PJSUA with a script so I was able to perform > automatically about 1500 calls. > > During a call each peer entity plays a reference speech sample that is > recorded on the other side of the call. Then a speech quality evaluation is > performed using a PESQ tool. > > The two machines hosting pjsua application are directly connected trough a > Ethernet switch in order to minimize the network impairment. > > > > The codec used during the call is G.711 and the expected PESQ score for > such codec is 4.40. > > The results of my tests highlight the speech quality is instable. That is, > often the PESQ score is the expected one but sometimes (about 10% of total > measures) the score is significatively less (3.50). > > Did anyone perform such type of tests or have experience of such type of > speech quality problems? > > > > Thanks for doing the tests and sharing the results. We also have PESQ tests as part of the automated unit tests framework (Python based, on pjsip-apps/src/test-pjsua directory), and got the report about intermittent audio degradation too. Our suspicion now lies with the jitter buffer. During the initial call establishment, perhaps due to high activity in the signaling thread, or the difference in the call establishment time between caller an callee, some RTP packets will be queued in the socket buffer and once the media is started these RTP packets will be stored in the jbuf in a burst. Often this burst exceeds the jbuf maximum size (it was 340ms), hence it will be discarded. At other times, some frames will also be discarded by the jbuf when it tries to optimize the latency. These discard operations will cause click noise in the playback, causing the PESQ score to degrade. That's probably the cause of your results. We have a ticket for this (http://trac.pjsip.org/repos/ticket/638) and Nanang has a pending commit for this ticket, hopefully the situation will improve by then. It will be good if you could retest again with the new changes then, to get a second opinion on this. How long did you set the call duration to? I suspect we will get better PESQ score if you run the call to longer duration, after the media is stabilized after the initial setup activity. Cheers Benny Best Regards > > Massimiliano Montevecchi >

IK

Ilya Kalinin

Mon, Sep 22, 2008 3:25 PM

Hi, again.

I trying to use alternative media stack with pjsip, I wonder I there is
a simple way to do it. How I can deactivate pjmedia and reattach new
stack without messing too much with pjsip code. I'm sorry if my question
is too general, but maybe there is something I need to know before
entering too deep into the code.

Thanks.

Hi, again. I trying to use alternative media stack with pjsip, I wonder I there is a simple way to do it. How I can deactivate pjmedia and reattach new stack without messing too much with pjsip code. I'm sorry if my question is too general, but maybe there is something I need to know before entering too deep into the code. Thanks.

NI

Nanang Izzuddin

Mon, Sep 22, 2008 6:25 PM

Hi,

Here is the patch (candidate) to be applied for ticket #638, we
haven't done enough tests on this actually. So it would be great if
you could help testing this patch and drop some feedbacks.

Thanks,
nanang

On Mon, Sep 22, 2008 at 9:56 PM, Benny Prijono bennylp@pjsip.org wrote:

On Mon, Sep 22, 2008 at 2:15 PM, Massimiliano Montevecchi
massimiliano.montevecchi@elsagdatamat.com wrote: