Multiple playback channels in pjsip?

TK
Thomas Kluge
Wed, Jan 21, 2009 10:28 AM

Hi all,

We try to build a VoIP application for simulating several audio media
features on top of the pjsip stack. This application should have the
following features related to the  pjsip-stack:

  • support multiple ports of a jack client
  • connection between transport streams and ports of a jack client (no
    mixing)

In my opinion there are two different approaches to a solution, every one
leads to several poblems I want to describe in relation to the model used.
I guess, that a solution would require significant changes/extensions
within the pjsip stack.

*** Recording model (suitable in both models): ***

Jack client x input port  - conferenc bridge master port - tx to all
transport streams (that's already the case)

*** Playback models: ***

Model 1


---====

  • use 1 jack input client and connect it to the conferenc bridge master
    port
  • use n-1 jack clients with 1 output port each and register them to
    different ports at the conference bridge.
  • That would require a support of n sounddevices simultaneously (todo).

Jack client 1 output port - conferenc bridge - rx transport stream 1
Jack client 2 output port - conferenc bridge - rx transport stream 2
Jack client 3 output port - conferenc bridge - rx transport stream 3
...
Jack client n-1 output port - conferenc bridge - rx transport stream n-1

Issues:

  • As I see there is currently only one sounddevice supported by the
    conderence bridge. This sounddevice has been selected within collected
    device information gathered from hostapi/portaudio.
  • Setting a new device will clear !! the previous one and register the new
    one to the master conference port.

Solution:

  • change the function pjsua_set_snd_dev() and related functions to handle
    more than one sounddevice at once
  • mirror the master port creation to other conference ports. (I don't know
    if this is possible using the same get_frame/put_frame callbacks of the
    master port.)
  • assign every sounddevice a separate conference port
  • Ensure that sounddevce functions like OpenStream() can be called on
    different sounddevices simultaneously

Model 2


---====

  • use a jack client with one input port and n-1 output ports and connect
    it to the conference bridge master port.
  • That would require support of multiple channels from PortAudio to
    conference bridge (todo)

Jack client x output port 1 - conferenc bridge - rx transport stream 1
Jack client x output port 2 - conferenc bridge - rx transport stream 2
Jack client x output port 3 - conferenc bridge - rx transport stream 3
...
Jack client x output port n-1 - conferenc bridge - rx transport stream n-1

Issues:

Starting from Jack I retraced the output port/channel information to the
conference bridge. I figured out that the information about multiple
ports/channels of a sounddevice gets lost in the transition from portaudio
(functions    NonAdaptingProcess/AdaptingOutputProcess) to internal
functions like PaPlayerCallback. It looks like only the outputbuffers of the
first port/channel will be processed within pjspip . (variable void *output,
array which holds per channel output-buffers on the PortAudio-side)

Solution:

  • provide, handle the channel / outputbuffer information from PortAudio to
    internal processing structures
  • disable mixing in conference bridge
  • connect output ports with transport streams (function get_buffer())
    (I have no idea how I can resolve this using the current structures).

Any thoughts about that, did anybody something similar before?

--
Thanks And Regards,
Thomas Kluge

Hi all, We try to build a VoIP application for simulating several audio media features on top of the pjsip stack. This application should have the following features related to the pjsip-stack: - support multiple ports of a jack client - connection between transport streams and ports of a jack client (no mixing) In my opinion there are two different approaches to a solution, every one leads to several poblems I want to describe in relation to the model used. I guess, that a solution would require significant changes/extensions within the pjsip stack. *** Recording model (suitable in both models): *** Jack client x input port - conferenc bridge master port - tx to all transport streams (that's already the case) *** Playback models: *** Model 1 ====================================================================== - use 1 jack input client and connect it to the conferenc bridge master port - use n-1 jack clients with 1 output port each and register them to different ports at the conference bridge. - That would require a support of n sounddevices simultaneously (todo). Jack client 1 output port - conferenc bridge - rx transport stream 1 Jack client 2 output port - conferenc bridge - rx transport stream 2 Jack client 3 output port - conferenc bridge - rx transport stream 3 ... Jack client n-1 output port - conferenc bridge - rx transport stream n-1 Issues: - As I see there is currently only one sounddevice supported by the conderence bridge. This sounddevice has been selected within collected device information gathered from hostapi/portaudio. - Setting a new device will clear !! the previous one and register the new one to the master conference port. Solution: - change the function pjsua_set_snd_dev() and related functions to handle more than one sounddevice at once - mirror the master port creation to other conference ports. (I don't know if this is possible using the same get_frame/put_frame callbacks of the master port.) - assign every sounddevice a separate conference port - Ensure that sounddevce functions like OpenStream() can be called on different sounddevices simultaneously Model 2 ====================================================================== - use a jack client with one input port and n-1 output ports and connect it to the conference bridge master port. - That would require support of multiple channels from PortAudio to conference bridge (todo) Jack client x output port 1 - conferenc bridge - rx transport stream 1 Jack client x output port 2 - conferenc bridge - rx transport stream 2 Jack client x output port 3 - conferenc bridge - rx transport stream 3 ... Jack client x output port n-1 - conferenc bridge - rx transport stream n-1 Issues: Starting from Jack I retraced the output port/channel information to the conference bridge. I figured out that the information about multiple ports/channels of a sounddevice gets lost in the transition from portaudio (functions NonAdaptingProcess/AdaptingOutputProcess) to internal functions like PaPlayerCallback. It looks like only the outputbuffers of the first port/channel will be processed within pjspip . (variable void *output, array which holds per channel output-buffers on the PortAudio-side) Solution: - provide, handle the channel / outputbuffer information from PortAudio to internal processing structures - disable mixing in conference bridge - connect output ports with transport streams (function get_buffer()) (I have no idea how I can resolve this using the current structures). Any thoughts about that, did anybody something similar before? -- Thanks And Regards, Thomas Kluge
BP
Benny Prijono
Wed, Jan 21, 2009 10:13 PM

Hi Thomas,

Sorry for the late reply.

If I read your post correctly, I think you're modeling the solution based on
the assumption that we can't use multiple sound devices with the conference
bridge. That's not true actually.

You can create another sound device (assuming PA Jack supports this), create
and connect a splitcomb to the snd dev, then create a rev_port from the
splitcomb, and register this rev_port to the bridge (just as you register
any other port to the bridge). There is a snippet for this in pjsua_app.c,
see the stereo_demo() function towards the bottom of the file.

Alternatively, since you don't need mixing, why don't you just disable the
bridge altogether, and connect the stream directly to the corresponding jack
sound dev? This is explained here:
http://trac.pjsip.org/repos/wiki/FAQ#pjsua-lib-perf

cheers
Benny

On Wed, Jan 21, 2009 at 10:28 AM, Thomas Kluge T.Kluge@gmx.com wrote:

Hi all,

We try to build a VoIP application for simulating several audio media
features on top of the pjsip stack. This application should have the
following features related to the  pjsip-stack:

  • support multiple ports of a jack client
  • connection between transport streams and ports of a jack client (no
    mixing)

In my opinion there are two different approaches to a solution, every one
leads to several poblems I want to describe in relation to the model used.
I guess, that a solution would require significant changes/extensions
within the pjsip stack.

*** Recording model (suitable in both models): ***

Jack client x input port  - conferenc bridge master port - tx to all
transport streams (that's already the case)

*** Playback models: ***

Model 1


---====

  • use 1 jack input client and connect it to the conferenc bridge master
    port
  • use n-1 jack clients with 1 output port each and register them to
    different ports at the conference bridge.
  • That would require a support of n sounddevices simultaneously (todo).

Jack client 1 output port - conferenc bridge - rx transport stream 1
Jack client 2 output port - conferenc bridge - rx transport stream 2
Jack client 3 output port - conferenc bridge - rx transport stream 3
...
Jack client n-1 output port - conferenc bridge - rx transport stream n-1

Issues:

  • As I see there is currently only one sounddevice supported by the
    conderence bridge. This sounddevice has been selected within collected
    device information gathered from hostapi/portaudio.
  • Setting a new device will clear !! the previous one and register the new
    one to the master conference port.

Solution:

  • change the function pjsua_set_snd_dev() and related functions to handle
    more than one sounddevice at once
  • mirror the master port creation to other conference ports. (I don't know
    if this is possible using the same get_frame/put_frame callbacks of the
    master port.)
  • assign every sounddevice a separate conference port
  • Ensure that sounddevce functions like OpenStream() can be called on
    different sounddevices simultaneously

Model 2


---====

  • use a jack client with one input port and n-1 output ports and connect
    it to the conference bridge master port.
  • That would require support of multiple channels from PortAudio to
    conference bridge (todo)

Jack client x output port 1 - conferenc bridge - rx transport stream 1
Jack client x output port 2 - conferenc bridge - rx transport stream 2
Jack client x output port 3 - conferenc bridge - rx transport stream 3
...
Jack client x output port n-1 - conferenc bridge - rx transport stream n-1

Issues:

Starting from Jack I retraced the output port/channel information to the
conference bridge. I figured out that the information about multiple
ports/channels of a sounddevice gets lost in the transition from portaudio
(functions    NonAdaptingProcess/AdaptingOutputProcess) to internal
functions like PaPlayerCallback. It looks like only the outputbuffers of
the
first port/channel will be processed within pjspip . (variable void
*output,
array which holds per channel output-buffers on the PortAudio-side)

Solution:

  • provide, handle the channel / outputbuffer information from PortAudio to
    internal processing structures
  • disable mixing in conference bridge
  • connect output ports with transport streams (function get_buffer())
    (I have no idea how I can resolve this using the current structures).

Any thoughts about that, did anybody something similar before?

--
Thanks And Regards,
Thomas Kluge


Visit our blog: http://blog.pjsip.org

pjsip mailing list
pjsip@lists.pjsip.org
http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org

Hi Thomas, Sorry for the late reply. If I read your post correctly, I think you're modeling the solution based on the assumption that we can't use multiple sound devices with the conference bridge. That's not true actually. You can create another sound device (assuming PA Jack supports this), create and connect a splitcomb to the snd dev, then create a rev_port from the splitcomb, and register this rev_port to the bridge (just as you register any other port to the bridge). There is a snippet for this in pjsua_app.c, see the stereo_demo() function towards the bottom of the file. Alternatively, since you don't need mixing, why don't you just disable the bridge altogether, and connect the stream directly to the corresponding jack sound dev? This is explained here: http://trac.pjsip.org/repos/wiki/FAQ#pjsua-lib-perf cheers Benny On Wed, Jan 21, 2009 at 10:28 AM, Thomas Kluge <T.Kluge@gmx.com> wrote: > Hi all, > > We try to build a VoIP application for simulating several audio media > features on top of the pjsip stack. This application should have the > following features related to the pjsip-stack: > > - support multiple ports of a jack client > - connection between transport streams and ports of a jack client (no > mixing) > > In my opinion there are two different approaches to a solution, every one > leads to several poblems I want to describe in relation to the model used. > I guess, that a solution would require significant changes/extensions > within the pjsip stack. > > *** Recording model (suitable in both models): *** > > Jack client x input port - conferenc bridge master port - tx to all > transport streams (that's already the case) > > *** Playback models: *** > > Model 1 > ====================================================================== > > - use 1 jack input client and connect it to the conferenc bridge master > port > - use n-1 jack clients with 1 output port each and register them to > different ports at the conference bridge. > - That would require a support of n sounddevices simultaneously (todo). > > Jack client 1 output port - conferenc bridge - rx transport stream 1 > Jack client 2 output port - conferenc bridge - rx transport stream 2 > Jack client 3 output port - conferenc bridge - rx transport stream 3 > ... > Jack client n-1 output port - conferenc bridge - rx transport stream n-1 > > Issues: > > - As I see there is currently only one sounddevice supported by the > conderence bridge. This sounddevice has been selected within collected > device information gathered from hostapi/portaudio. > - Setting a new device will clear !! the previous one and register the new > one to the master conference port. > > Solution: > > - change the function pjsua_set_snd_dev() and related functions to handle > more than one sounddevice at once > - mirror the master port creation to other conference ports. (I don't know > if this is possible using the same get_frame/put_frame callbacks of the > master port.) > - assign every sounddevice a separate conference port > - Ensure that sounddevce functions like OpenStream() can be called on > different sounddevices simultaneously > > > Model 2 > ====================================================================== > > - use a jack client with one input port and n-1 output ports and connect > it to the conference bridge master port. > - That would require support of multiple channels from PortAudio to > conference bridge (todo) > > > Jack client x output port 1 - conferenc bridge - rx transport stream 1 > Jack client x output port 2 - conferenc bridge - rx transport stream 2 > Jack client x output port 3 - conferenc bridge - rx transport stream 3 > ... > Jack client x output port n-1 - conferenc bridge - rx transport stream n-1 > > Issues: > > Starting from Jack I retraced the output port/channel information to the > conference bridge. I figured out that the information about multiple > ports/channels of a sounddevice gets lost in the transition from portaudio > (functions NonAdaptingProcess/AdaptingOutputProcess) to internal > functions like PaPlayerCallback. It looks like only the outputbuffers of > the > first port/channel will be processed within pjspip . (variable void > *output, > array which holds per channel output-buffers on the PortAudio-side) > > Solution: > > - provide, handle the channel / outputbuffer information from PortAudio to > internal processing structures > - disable mixing in conference bridge > - connect output ports with transport streams (function get_buffer()) > (I have no idea how I can resolve this using the current structures). > > > Any thoughts about that, did anybody something similar before? > > -- > Thanks And Regards, > Thomas Kluge > > _______________________________________________ > Visit our blog: http://blog.pjsip.org > > pjsip mailing list > pjsip@lists.pjsip.org > http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org >
NY
Noga Yehudai
Thu, Jan 22, 2009 6:46 AM

Hi,

I hope you could help me here, I have 2 questions:

  1. Did anyone implement the RTCP-XR in PJSUA API?

  2. In the documentation it says: "PJMEDIA implements subsets of RTCP XR
    specification"

  • What were the parameters that were implemented, and what was left out

Thanks!

Noga

Hi, I hope you could help me here, I have 2 questions: 1. Did anyone implement the RTCP-XR in PJSUA API? 2. In the documentation it says: "PJMEDIA implements subsets of RTCP XR specification" - What were the parameters that were implemented, and what was left out Thanks! Noga
BP
Benny Prijono
Thu, Jan 22, 2009 8:09 AM

On Thu, Jan 22, 2009 at 6:46 AM, Noga Yehudai nogab@juniper.net wrote:

Hi,

I hope you could help me here, I have 2 questions:

  1. Did anyone implement the RTCP-XR in PJSUA API?
  1. In the documentation it says: "PJMEDIA implements subsets of RTCP XR
    specification"

– What were the parameters that were implemented, and what was left out

On Thu, Jan 22, 2009 at 6:46 AM, Noga Yehudai <nogab@juniper.net> wrote: > Hi, > > I hope you could help me here, I have 2 questions: > > 1. Did anyone implement the RTCP-XR in PJSUA API? > > Yes: http://www.pjsip.org/pjmedia/docs/html/group__PJMED__RTCP__XR.htm > 2. In the documentation it says: "PJMEDIA implements subsets of RTCP XR > specification" > > – What were the parameters that were implemented, and what was left out > > > http://trac.pjsip.org/repos/ticket/513 cheers Benny > > Thanks! > > Noga > > _______________________________________________ > Visit our blog: http://blog.pjsip.org > > pjsip mailing list > pjsip@lists.pjsip.org > http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org > >
NY
Noga Yehudai
Thu, Jan 22, 2009 8:50 AM

Benny hi,

Thanks for the prompt response!

Noga


From: pjsip-bounces@lists.pjsip.org
[mailto:pjsip-bounces@lists.pjsip.org] On Behalf Of Benny Prijono
Sent: Thursday, January 22, 2009 10:10 AM
To: pjsip list
Subject: Re: [pjsip] RTCP-XR

On Thu, Jan 22, 2009 at 6:46 AM, Noga Yehudai nogab@juniper.net wrote:

Hi,

I hope you could help me here, I have 2 questions:

1. Did anyone implement the RTCP-XR in PJSUA API? 

Yes: http://www.pjsip.org/pjmedia/docs/html/group__PJMED__RTCP__XR.htm

2. In the documentation it says: "PJMEDIA implements subsets of

RTCP XR specification"

- What were the parameters that were implemented, and what was

left out

http://trac.pjsip.org/repos/ticket/513

cheers
Benny

Thanks!

Noga


_______________________________________________
Visit our blog: http://blog.pjsip.org

pjsip mailing list
pjsip@lists.pjsip.org
http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org
Benny hi, Thanks for the prompt response! Noga ________________________________ From: pjsip-bounces@lists.pjsip.org [mailto:pjsip-bounces@lists.pjsip.org] On Behalf Of Benny Prijono Sent: Thursday, January 22, 2009 10:10 AM To: pjsip list Subject: Re: [pjsip] RTCP-XR On Thu, Jan 22, 2009 at 6:46 AM, Noga Yehudai <nogab@juniper.net> wrote: Hi, I hope you could help me here, I have 2 questions: 1. Did anyone implement the RTCP-XR in PJSUA API? Yes: http://www.pjsip.org/pjmedia/docs/html/group__PJMED__RTCP__XR.htm 2. In the documentation it says: "PJMEDIA implements subsets of RTCP XR specification" - What were the parameters that were implemented, and what was left out http://trac.pjsip.org/repos/ticket/513 cheers Benny Thanks! Noga _______________________________________________ Visit our blog: http://blog.pjsip.org pjsip mailing list pjsip@lists.pjsip.org http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org