memory usage of epoll implementation

SC
Simon Chen
Mon, Aug 18, 2008 9:09 AM

Hi!

In ioqueue_select.c, memory of pj_ioqueue_key_t will be reused when compiler option "PJ_IOQUEUE_HAS_SAFE_UNREG" is set. But in ioqueue_epoll.c, it doesn't support this. 

In pj_ioqueue_register_sock(), it allocates the memory and creates a mutex for pj_ioqueue_key_t by pj_pool_zalloc. But these memory will not be released before memory poll is released. This will increase the memory usage after calling pj_ioqueue_register_sock() in ioqueue_epoll.c. Can ioqueue_epoll.c add the same mechanism to avoid the memory increase? Is there any other concern? Thanks!

Simon Chen

Hi! In ioqueue_select.c, memory of pj_ioqueue_key_t will be reused when compiler option "PJ_IOQUEUE_HAS_SAFE_UNREG" is set. But in ioqueue_epoll.c, it doesn't support this. In pj_ioqueue_register_sock(), it allocates the memory and creates a mutex for pj_ioqueue_key_t by pj_pool_zalloc. But these memory will not be released before memory poll is released. This will increase the memory usage after calling pj_ioqueue_register_sock() in ioqueue_epoll.c. Can ioqueue_epoll.c add the same mechanism to avoid the memory increase? Is there any other concern? Thanks! Simon Chen
BP
Benny Prijono
Mon, Aug 18, 2008 11:47 AM

On Mon, Aug 18, 2008 at 10:09 AM, Simon Chen simonmychen@seed.net.twwrote:

Hi!

 In ioqueue_select.c, memory of pj_ioqueue_key_t will be reused when

compiler option "PJ_IOQUEUE_HAS_SAFE_UNREG" is set. But in
ioqueue_epoll.c, it doesn't support this.

 In pj_ioqueue_register_sock(), it allocates the memory and creates a

mutex for pj_ioqueue_key_t by pj_pool_zalloc. But these memory will not be
released before memory poll is released. This will increase the memory
usage after calling pj_ioqueue_register_sock() in ioqueue_epoll.c. Can
ioqueue_epoll.c add the same mechanism to avoid the memory increase? Is
there any other concern? Thanks!

In general, the epoll ioqueue backend is not actively maintained lately,
hence it's not even offered as configuration choice. So I'm not surprised
that it has some holes in it. If you think you can make it work, feel free
to suggest a patch, I'd be happy to review it. :)

Cheers
Benny

Simon Chen

On Mon, Aug 18, 2008 at 10:09 AM, Simon Chen <simonmychen@seed.net.tw>wrote: > Hi! > > In ioqueue_select.c, memory of pj_ioqueue_key_t will be reused when > compiler option "PJ_IOQUEUE_HAS_SAFE_UNREG" is set. But in > ioqueue_epoll.c, it doesn't support this. > > In pj_ioqueue_register_sock(), it allocates the memory and creates a > mutex for pj_ioqueue_key_t by pj_pool_zalloc. But these memory will not be > released before memory poll is released. This will increase the memory > usage after calling pj_ioqueue_register_sock() in ioqueue_epoll.c. Can > ioqueue_epoll.c add the same mechanism to avoid the memory increase? Is > there any other concern? Thanks! > > In general, the epoll ioqueue backend is not actively maintained lately, hence it's not even offered as configuration choice. So I'm not surprised that it has some holes in it. If you think you can make it work, feel free to suggest a patch, I'd be happy to review it. :) Cheers Benny > Simon Chen > >
SC
Simon Chen
Wed, Aug 20, 2008 9:29 AM

Benny:

Thanks! I will try to implement it. Which test code can I use to verify my implementation in development?

Simon
----- Original Message -----
From: Benny Prijono
To: pjsip list
Sent: Monday, August 18, 2008 7:47 PM
Subject: Re: [pjsip] memory usage of epoll implementation

On Mon, Aug 18, 2008 at 10:09 AM, Simon Chen simonmychen@seed.net.tw wrote:

Hi!

    In ioqueue_select.c, memory of pj_ioqueue_key_t will be reused when compiler option "PJ_IOQUEUE_HAS_SAFE_UNREG" is set. But in ioqueue_epoll.c, it doesn't support this. 

    In pj_ioqueue_register_sock(), it allocates the memory and creates a mutex for pj_ioqueue_key_t by pj_pool_zalloc. But these memory will not be released before memory poll is released. This will increase the memory usage after calling pj_ioqueue_register_sock() in ioqueue_epoll.c. Can ioqueue_epoll.c add the same mechanism to avoid the memory increase? Is there any other concern? Thanks!

In general, the epoll ioqueue backend is not actively maintained lately, hence it's not even offered as configuration choice. So I'm not surprised that it has some holes in it. If you think you can make it work, feel free to suggest a patch, I'd be happy to review it. :)

Cheers
Benny

Simon Chen


Visit our blog: http://blog.pjsip.org

pjsip mailing list
pjsip@lists.pjsip.org
http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org

Benny: Thanks! I will try to implement it. Which test code can I use to verify my implementation in development? Simon ----- Original Message ----- From: Benny Prijono To: pjsip list Sent: Monday, August 18, 2008 7:47 PM Subject: Re: [pjsip] memory usage of epoll implementation On Mon, Aug 18, 2008 at 10:09 AM, Simon Chen <simonmychen@seed.net.tw> wrote: Hi! In ioqueue_select.c, memory of pj_ioqueue_key_t will be reused when compiler option "PJ_IOQUEUE_HAS_SAFE_UNREG" is set. But in ioqueue_epoll.c, it doesn't support this. In pj_ioqueue_register_sock(), it allocates the memory and creates a mutex for pj_ioqueue_key_t by pj_pool_zalloc. But these memory will not be released before memory poll is released. This will increase the memory usage after calling pj_ioqueue_register_sock() in ioqueue_epoll.c. Can ioqueue_epoll.c add the same mechanism to avoid the memory increase? Is there any other concern? Thanks! In general, the epoll ioqueue backend is not actively maintained lately, hence it's not even offered as configuration choice. So I'm not surprised that it has some holes in it. If you think you can make it work, feel free to suggest a patch, I'd be happy to review it. :) Cheers Benny Simon Chen ------------------------------------------------------------------------------ _______________________________________________ Visit our blog: http://blog.pjsip.org pjsip mailing list pjsip@lists.pjsip.org http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org
BP
Benny Prijono
Wed, Aug 20, 2008 9:34 AM

On Wed, Aug 20, 2008 at 10:29 AM, Simon Chen simonmychen@seed.net.twwrote:

Benny:

 Thanks! I will try to implement it. Which test code can I use to

verify my implementation in development?

Cool. You can test it with pjlib-test, then use pjsua or your application to
see if it works.

Cheers
Benny

On Wed, Aug 20, 2008 at 10:29 AM, Simon Chen <simonmychen@seed.net.tw>wrote: > Benny: > > Thanks! I will try to implement it. Which test code can I use to > verify my implementation in development? > > Cool. You can test it with pjlib-test, then use pjsua or your application to see if it works. Cheers Benny
SC
Simon Chen
Fri, Sep 5, 2008 8:27 AM

Hi! Benny,

I have tried to add PJ_IOQUEUE_HAS_SAFE_UNREG mechanism to ioqueue_epoll.c. I refer the ioqueue_select.c to modify it and test with my own program. The memory usage will not increase and work ok. I put the modified ioqueue_epoll.c in the attachment. Can you help me to double check it? Thanks! 

P.S.    I find the ioqueue_winnt.c in 1.0 RC1 adding "concurrency" mechanism. Does the ioqueue_epoll.c need to add the same mechanism?

Simon

----- Original Message -----
From: Benny Prijono
To: pjsip list
Sent: Monday, August 18, 2008 7:47 PM
Subject: Re: [pjsip] memory usage of epoll implementation

On Mon, Aug 18, 2008 at 10:09 AM, Simon Chen simonmychen@seed.net.tw wrote:

Hi!

    In ioqueue_select.c, memory of pj_ioqueue_key_t will be reused when compiler option "PJ_IOQUEUE_HAS_SAFE_UNREG" is set. But in ioqueue_epoll.c, it doesn't support this. 

    In pj_ioqueue_register_sock(), it allocates the memory and creates a mutex for pj_ioqueue_key_t by pj_pool_zalloc. But these memory will not be released before memory poll is released. This will increase the memory usage after calling pj_ioqueue_register_sock() in ioqueue_epoll.c. Can ioqueue_epoll.c add the same mechanism to avoid the memory increase? Is there any other concern? Thanks!

In general, the epoll ioqueue backend is not actively maintained lately, hence it's not even offered as configuration choice. So I'm not surprised that it has some holes in it. If you think you can make it work, feel free to suggest a patch, I'd be happy to review it. :)

Cheers
Benny

Simon Chen


Visit our blog: http://blog.pjsip.org

pjsip mailing list
pjsip@lists.pjsip.org
http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org

Hi! Benny, I have tried to add PJ_IOQUEUE_HAS_SAFE_UNREG mechanism to ioqueue_epoll.c. I refer the ioqueue_select.c to modify it and test with my own program. The memory usage will not increase and work ok. I put the modified ioqueue_epoll.c in the attachment. Can you help me to double check it? Thanks! P.S. I find the ioqueue_winnt.c in 1.0 RC1 adding "concurrency" mechanism. Does the ioqueue_epoll.c need to add the same mechanism? Simon ----- Original Message ----- From: Benny Prijono To: pjsip list Sent: Monday, August 18, 2008 7:47 PM Subject: Re: [pjsip] memory usage of epoll implementation On Mon, Aug 18, 2008 at 10:09 AM, Simon Chen <simonmychen@seed.net.tw> wrote: Hi! In ioqueue_select.c, memory of pj_ioqueue_key_t will be reused when compiler option "PJ_IOQUEUE_HAS_SAFE_UNREG" is set. But in ioqueue_epoll.c, it doesn't support this. In pj_ioqueue_register_sock(), it allocates the memory and creates a mutex for pj_ioqueue_key_t by pj_pool_zalloc. But these memory will not be released before memory poll is released. This will increase the memory usage after calling pj_ioqueue_register_sock() in ioqueue_epoll.c. Can ioqueue_epoll.c add the same mechanism to avoid the memory increase? Is there any other concern? Thanks! In general, the epoll ioqueue backend is not actively maintained lately, hence it's not even offered as configuration choice. So I'm not surprised that it has some holes in it. If you think you can make it work, feel free to suggest a patch, I'd be happy to review it. :) Cheers Benny Simon Chen ------------------------------------------------------------------------------ _______________________________________________ Visit our blog: http://blog.pjsip.org pjsip mailing list pjsip@lists.pjsip.org http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org
BP
Benny Prijono
Fri, Sep 5, 2008 2:05 PM

On Fri, Sep 5, 2008 at 10:27 AM, Simon Chen simonmychen@seed.net.tw wrote:

Hi! Benny,

 I have tried to add PJ_IOQUEUE_HAS_SAFE_UNREG mechanism to

ioqueue_epoll.c. I refer the ioqueue_select.c to modify it and test with my
own program. The memory usage will not increase and work ok. I put the
modified ioqueue_epoll.c in the attachment. Can you help me to double check
it? Thanks!

Thanks for the patch.

P.S.    I find the ioqueue_winnt.c in 1.0 RC1 adding "concurrency"

mechanism. Does the ioqueue_epoll.c need to add the same mechanism?

Yeah I think so.

Cheers
Benny

Simon

----- Original Message -----
From: Benny Prijono bennylp@pjsip.org
To: pjsip list pjsip@lists.pjsip.org
Sent: Monday, August 18, 2008 7:47 PM
Subject: Re: [pjsip] memory usage of epoll implementation

On Mon, Aug 18, 2008 at 10:09 AM, Simon Chen simonmychen@seed.net.twwrote:

Hi!

 In ioqueue_select.c, memory of pj_ioqueue_key_t will be reused when

compiler option "PJ_IOQUEUE_HAS_SAFE_UNREG" is set. But in
ioqueue_epoll.c, it doesn't support this.

 In pj_ioqueue_register_sock(), it allocates the memory and creates a

mutex for pj_ioqueue_key_t by pj_pool_zalloc. But these memory will not be
released before memory poll is released. This will increase the memory
usage after calling pj_ioqueue_register_sock() in ioqueue_epoll.c. Can
ioqueue_epoll.c add the same mechanism to avoid the memory increase? Is
there any other concern? Thanks!

In general, the epoll ioqueue backend is not actively maintained lately,
hence it's not even offered as configuration choice. So I'm not surprised
that it has some holes in it. If you think you can make it work, feel free
to suggest a patch, I'd be happy to review it. :)

Cheers
Benny

Simon Chen

On Fri, Sep 5, 2008 at 10:27 AM, Simon Chen <simonmychen@seed.net.tw> wrote: > Hi! Benny, > > > I have tried to add PJ_IOQUEUE_HAS_SAFE_UNREG mechanism to > ioqueue_epoll.c. I refer the ioqueue_select.c to modify it and test with my > own program. The memory usage will not increase and work ok. I put the > modified ioqueue_epoll.c in the attachment. Can you help me to double check > it? Thanks! > > Thanks for the patch. P.S. I find the ioqueue_winnt.c in 1.0 RC1 adding "concurrency" > mechanism. Does the ioqueue_epoll.c need to add the same mechanism? > > Yeah I think so. Cheers Benny > Simon > > > ----- Original Message ----- > *From:* Benny Prijono <bennylp@pjsip.org> > *To:* pjsip list <pjsip@lists.pjsip.org> > *Sent:* Monday, August 18, 2008 7:47 PM > *Subject:* Re: [pjsip] memory usage of epoll implementation > > On Mon, Aug 18, 2008 at 10:09 AM, Simon Chen <simonmychen@seed.net.tw>wrote: > >> Hi! >> >> In ioqueue_select.c, memory of pj_ioqueue_key_t will be reused when >> compiler option "PJ_IOQUEUE_HAS_SAFE_UNREG" is set. But in >> ioqueue_epoll.c, it doesn't support this. >> >> In pj_ioqueue_register_sock(), it allocates the memory and creates a >> mutex for pj_ioqueue_key_t by pj_pool_zalloc. But these memory will not be >> released before memory poll is released. This will increase the memory >> usage after calling pj_ioqueue_register_sock() in ioqueue_epoll.c. Can >> ioqueue_epoll.c add the same mechanism to avoid the memory increase? Is >> there any other concern? Thanks! >> >> > > In general, the epoll ioqueue backend is not actively maintained lately, > hence it's not even offered as configuration choice. So I'm not surprised > that it has some holes in it. If you think you can make it work, feel free > to suggest a patch, I'd be happy to review it. :) > > Cheers > Benny > > > >> Simon Chen >> >> > > ------------------------------ > > _______________________________________________ > Visit our blog: http://blog.pjsip.org > > pjsip mailing list > pjsip@lists.pjsip.org > http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org > > > _______________________________________________ > Visit our blog: http://blog.pjsip.org > > pjsip mailing list > pjsip@lists.pjsip.org > http://lists.pjsip.org/mailman/listinfo/pjsip_lists.pjsip.org > >