[USRP-users] rx_streamer->recv

The Tilla tilla at comcast.net
Tue Jul 21 08:14:02 EDT 2015


Hi Neel,

 

I know I am in the 0.000001 percent, but have you had a chance to make any progress with the slowdown?

 

Thanks,

 

From: USRP-users [mailto:usrp-users-bounces at lists.ettus.com] On Behalf Of tilla--- via USRP-users
Sent: Thursday, July 09, 2015 10:17 AM
To: Neel Pandeya; usrp-users
Subject: Re: [USRP-users] rx_streamer->recv

 

Hi Neel,

 

Attached is a sample application that reproduces the problem.

 

It takes command line argument for usrp config string, please use the following:

 

addr=YOUR_IP_ADDR,recv_frame_size=1044

 

This app is hard coded to the following specs:

 

900 MHz freq

10 MSA/sec

2 MHz bandwidth

applications recv buffer size = 32 samples

 

My software platform is

 

win7 SP1 64 bit

Visual Studio 2013 update 4

UHD 3.8.3 64 bit

Boost 1.56

 

The app is coded to print out a message any time recv takes longer than 1 millisecond using boost timers.  This should never happen as the recv frame size is set to 256 samples, which at 10 MSa/sec sampling rate is a packet arriving every 25.6 + slop usec, on average.  So in 1 millisecond, somewhere around 40 packets will have arrived to the NIC.

 

If you move the logging down into UHD to the line I identified previously ( info.buff.reset() ), you will see that this is where the slowdown takes place periodically.





I havent investigated into that call, I leave that up to the experts :)

 

As I said previously, NIC is tuned, things work properly at all other times, not the virus scanner or any govenors, fastdatagram set to 4088 (but for this packet size, is not relevant).

 

Please let me know if I can provide any more details.

 

Thanks,

 

 

  _____  

From: "The Tilla via USRP-users" <usrp-users at lists.ettus.com>
To: "Neel Pandeya" <neel.pandeya at ettus.com>
Cc: "usrp-users" <usrp-users at lists.ettus.com>
Sent: Wednesday, June 17, 2015 10:34:21 PM
Subject: Re: [USRP-users] rx_streamer->recv

 

 

More information:

 

The “slowdown” every second on calls to recv is *always* there…

 

At 1 GB memory utilization it is 100 usec “pause” every second

At 10 GB memory utilization it is 2 millisecond “pause” every second

At 20 GB memory utilization it is 3 milli-seconds “pause” every second

 

So it seems to increase with increasing memory utilization at the call to info.buff.reset()

 

The results of todays analysis J

 

From: Neel Pandeya [mailto:neel.pandeya at ettus.com] 
Sent: Sunday, June 14, 2015 11:01 AM
To: The Tilla
Cc: usrp-users at lists.ettus.com; Garey, Marshall Owen
Subject: Re: [USRP-users] rx_streamer->recv

 

Hello Tilla:

It looks like you're tried pretty much everything I can think of. I'm impressed that you have been able to get 99.9% good sample delivery in Windows 7. These infrequent, last-mile problems are the hardest to solve. At this point, I can only offer of the following suggestions.

As Marshall Owen Garey mentioned, I have heard many times anecdotally from customers that people are able to achieve better performance under Linux than Windows. As a test of your hardware, perhaps try running your application under Linux. An easy way to do this would be use a Live USB image [1,2].

I have also heard from two customers who were having performance problems on Windows 7 that they were able to improve performance by upgrading to Windows 8. I'm not sure if this is an option for you, but you might want to try this.



 

Have you made any tweaks to the registry? The only specific modification that we suggest is [3].

Which version of Boost are you using?

Have you enabled any optimization or processing off-loading features of your NIC?

Is there any other intermittent spike or burst in network traffic, perhaps on a different port such as one for your corporate network, that could be causing intermittent spikes in CPU load?

 

Certainly make sure that you have any background processes/services turn off, such as anti-virus, update tools, firewall, etc.


[1] http://files.ettus.com/liveusb/3.0/



 

[2] https://gnuradio.org/redmine/projects/gnuradio/wiki/GNURadioLiveDVD



 

[3] https://github.com/EttusResearch/uhd/blob/master/host/utils/FastSendDatagramThreshold.reg

--Neel



 

 

On 10 June 2015 at 16:25, Garey,

​​

Marshall Owen via USRP-users <usrp-users at lists.ettus.com> wrote:

I have had a very similar problem with the same setup (Windows 7 64-bit, N210, VS2013), and when I moved to Linux (which was a real pain) I was able to recv at a higher rate with no problems. So I agree with you – it’s probably Windows. The only suggestions I have is to try turning off any background programs and processes that you can (such as the antivirus). You could try the LiveUSB image provided by Ettus Research to see if it is a Windows problem (that’s what I did).

 

From: USRP-users [mailto:usrp-users-bounces at lists.ettus.com] On Behalf Of The Tilla via USRP-users
Sent: Wednesday, June 10, 2015 4:31 PM
To: usrp-users at lists.ettus.com
Subject: [USRP-users] rx_streamer->recv

 

Platform:

 

Win7 64 bit

N210

Visual Studio 2013

 

So may application is very sensitive to latencies.

 

I have taken 42 million measurements during runtime.  41+ million calls to recv execute in under 10 micro-seconds.  I am asking for 32 samples with packet size of 256 samples and sampling rate of 10 MSA/sec.

 

Sample delivery is pretty awesome 99.999 percent of the time…

 

Then there are the times when recv takes very close to 1 millisecond to return ( ~5000 times out of 42 million (yeah, pretty crazy) ).

There is never a time where no packets are delivered within 1 millisecond given a 10 MSA/sec sampling rate…

 

What this feels like is good old windows scheduler putting something in a state where it will never come back on to the processor until around 1 millisecond later no matter what interrupt is raised.

A timer was placed right before rx_streamer->recv and right after rx_streamer->recv to take measurements, no other work being done during timer measurement.

 

The only think that I could think of is the udp recv way down at the bottom of the stack…

 

But it works “properly” sooooo many times without blocking for a long period of time.

 

Things I have tried:

                Running process @ realtime priority

                Mucking with recv sample sizes, packet sizes, etc.

                Already disabled interrupt moderation and related settings

                Used windows server setting for process quantum

                Compiled with full optimizations

                Recv is in its own thread and processing around recv is very fast

                Affinitized application recv processing thread ( not UHD pirate thread, YAR! ) to CPU directly attached to NIC PCI card

                                NIC is Intel ET2 quad port card

 

Any suggestions?

 

Any OS settings to lower what feels like some sort of minimum a sleep/block interval that anyone is aware of?

 

Thanks

 

 


_______________________________________________
USRP-users mailing list
USRP-users at lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

 


_______________________________________________
USRP-users mailing list
USRP-users at lists.ettus.com
http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/attachments/20150721/ad78104e/attachment-0002.html>


More information about the USRP-users mailing list