[USRP-users] UHD 3.10.2 | X300 - High CPU load even for low samples rate

Michael West michael.west at ettus.com
Tue Oct 24 14:57:34 EDT 2017


Hi Kai,

One more thing.  To reduce the CPU load, you can try playing around with
the timeout parameter on this call:
https://github.com/EttusResearch/uhd/blob/maint/host/lib/usrp/device3/device3_io_impl.cpp#L357

Try increasing it from 1us to something like 10ms or more.  That controls
the timeout for the poll() call when waiting for a flow control packet.  On
the To Do list is to make that timeout value a function of the TX rate so
lower rates will have reduced CPU load and higher rates will be more
responsive.

Regards,
Michael

On Tue, Oct 24, 2017 at 10:19 AM, Michael West <michael.west at ettus.com>
wrote:

> Hi Kai,
>
> The increased CPU usage is expected.  It was done intentionally to make
> the UHD code more responsive to flow control messages in order to avoid
> underruns at higher sample rates.  Appropriate yields were put in place to
> avoid starvation.  What is not expected are the dropped packet (D) and
> receive timeout errors.  Can you reproduce the errors by running just the
> RX side (./benchmark_rate --args="address=192.168.10.2" --rx_rate 20e6)?
>
> Regards,
> Michael
>
> On Tue, Oct 24, 2017 at 5:40 AM, Kai-Uwe Storek via USRP-users <
> usrp-users at lists.ettus.com> wrote:
>
>> Is there something else I can in order to support the investigation of
>> the problem? Is it useful to open a new issue on github?
>>
>> 2017-10-20 19:23 GMT+02:00 Marcus D. Leech via USRP-users
>> <usrp-users at lists.ettus.com>:
>> > On 10/20/2017 04:20 AM, Kai-Uwe Storek via USRP-users wrote:
>> >>
>> >> Hey Marcus and Brent,
>> >>
>> >> thanks for suggestions. I'm not working in a VM.
>> >>
>> >> The benchmark tool underpins the behaviour:
>> >>
>> >> If I use the UHD version of the debian distribution, everything seems
>> >> normal:
>> >
>> > Interesting.  This is definitely something to be followed up on.
>> >
>> > I took a closer look at my own FG in this context, and indeed one of the
>> > CPUs is completely pegged.
>> >
>> >
>> >
>> >>
>> >> ------------------------------------------
>> >> /usr/lib/uhd/examples % ./benchmark_rate --args="address=192.168.10.2"
>> >> --rx_rate 20e6 --tx_rate 20e6
>> >> linux; GNU C++ version 6.3.0 20170221; Boost_106200;
>> >> UHD_003.009.005-0-unknown
>> >>
>> >>
>> >> Creating the usrp device with: address=192.168.10.2...
>> >> -- X300 initialization sequence...
>> >> -- Determining maximum frame size... 1472 bytes.
>> >> -- Setup basic communication...
>> >> -- Loading values from EEPROM...
>> >> -- Setup RF frontend clocking...
>> >> -- Radio 1x clock:200
>> >> -- Initialize Radio0 control...
>> >> -- Performing register loopback test... pass
>> >> -- Initialize Radio1 control...
>> >> -- Performing register loopback test... pass
>> >> Using Device: Single USRP:
>> >>    Device: X-Series Device
>> >>    Mboard 0: X300
>> >>    RX Channel: 0
>> >>      RX DSP: 0
>> >>      RX Dboard: A
>> >>      RX Subdev: WBXv3 RX+GDB
>> >>    RX Channel: 1
>> >>      RX DSP: 1
>> >>      RX Dboard: B
>> >>      RX Subdev: WBXv3 RX+GDB
>> >>    TX Channel: 0
>> >>      TX DSP: 0
>> >>      TX Dboard: A
>> >>      TX Subdev: WBXv3 TX+GDB
>> >>    TX Channel: 1
>> >>      TX DSP: 1
>> >>      TX Dboard: B
>> >>      TX Subdev: WBXv3 TX+GDB
>> >>
>> >> Setting device timestamp to 0...
>> >> Testing receive rate 20.000000 Msps on 1 channels
>> >> Testing transmit rate 20.000000 Msps on 1 channels
>> >> UUU
>> >> Benchmark rate summary:
>> >>    Num received samples:    200137060
>> >>    Num dropped samples:     0
>> >>    Num overflows detected:  0
>> >>    Num transmitted samples: 200114096
>> >>    Num sequence errors:     0
>> >>    Num underflows detected: 3
>> >>    Num late commands:       0
>> >>    Num timeouts:            0
>> >>
>> >>
>> >> Done!
>> >> ------------------------------------------
>> >>
>> >>
>> >> If I use the the maint version:
>> >>
>> >> ------------------------------------------
>> >> ~/gr_prefixes/stable/lib/uhd/examples % ./benchmark_rate
>> >> --args="address=192.168.10.2" --rx_rate 20e6 --tx_rate 20e6
>> >> linux; GNU C++ version 6.3.0 20170516; Boost_106200;
>> >> UHD_003.010.002.000-3-g122bfae1
>> >>
>> >>
>> >> Creating the usrp device with: address=192.168.10.2...
>> >> -- X300 initialization sequence...
>> >> -- Determining maximum frame size... 1472 bytes.
>> >> -- Setup basic communication...
>> >> -- Loading values from EEPROM...
>> >> -- Setup RF frontend clocking...
>> >> -- Radio 1x clock:200
>> >> -- Detecting internal GPSDO.... No GPSDO found
>> >> -- [DMA FIFO] Running BIST for FIFO 0... pass (Throughput: 1296.0MB/s)
>> >> -- [DMA FIFO] Running BIST for FIFO 1... pass (Throughput: 1299.4MB/s)
>> >> -- [RFNoC Radio] Performing register loopback test... pass
>> >> -- [RFNoC Radio] Performing register loopback test... pass
>> >> -- [RFNoC Radio] Performing register loopback test... pass
>> >> -- [RFNoC Radio] Performing register loopback test... pass
>> >> -- Performing timer loopback test... pass
>> >> -- Performing timer loopback test... pass
>> >> Using Device: Single USRP:
>> >>    Device: X-Series Device
>> >>    Mboard 0: X300
>> >>    RX Channel: 0
>> >>      RX DSP: 0
>> >>      RX Dboard: A
>> >>      RX Subdev: WBXv3 RX+GDB
>> >>    RX Channel: 1
>> >>      RX DSP: 0
>> >>      RX Dboard: B
>> >>      RX Subdev: WBXv3 RX+GDB
>> >>    TX Channel: 0
>> >>      TX DSP: 0
>> >>      TX Dboard: A
>> >>      TX Subdev: WBXv3 TX+GDB
>> >>    TX Channel: 1
>> >>      TX DSP: 0
>> >>      TX Dboard: B
>> >>      TX Subdev: WBXv3 TX+GDB
>> >>
>> >> Setting device timestamp to 0...
>> >> Testing receive rate 20.000000 Msps on 1 channels
>> >> Testing transmit rate 20.000000 Msps on 1 channels
>> >> UUUUReceiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> UDOReceiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >>
>> >> UHD Error:
>> >>      x300 fw communication failure #1
>> >>      EnvironmentError: IOError: x300 fw poke32 - reply timed out
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >>
>> >> UHD Error:
>> >>      x300 fw communication failure #2
>> >>      EnvironmentError: IOError: x300 fw poke32 - reply timed out
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >> Receiver error: ERROR_CODE_TIMEOUT, continuing...
>> >>
>> >> Benchmark rate summary:
>> >>    Num received samples:    119843251
>> >>    Num dropped samples:     3194618
>> >>    Num overflows detected:  1
>> >>    Num transmitted samples: 175807996
>> >>    Num sequence errors:     0
>> >>    Num underflows detected: 5
>> >>    Num late commands:       0
>> >>    Num timeouts:            39
>> >>
>> >>
>> >> Done!
>> >> ------------------------------------------
>> >>
>> >>
>> >> Between these tests I did nothing then flashing the X300 with the
>> >> appropriate image and doing a power cycle.
>> >>
>> >>
>> >>
>> >> @Marcus:
>> >> If I use your flowgraph mit UHD 3.9.5, my CPU load is very modest. No
>> >> core higher than 15%. Executing the graph with UHD 3.10.2 causes again
>> >> 100% CPU load on a single core over the entire time.
>> >>
>> >>
>> >>
>> >>
>> >>
>> >> 2017-10-19 23:05 GMT+02:00 Marcus D. Leech via USRP-users
>> >> <usrp-users at lists.ettus.com>:
>> >>>
>> >>> On 10/19/2017 10:25 AM, Kai-Uwe Storek via USRP-users wrote:
>> >>>>
>> >>>> Hey,
>> >>>>
>> >>>> after experiencing some problems (time outs, etc) with the current
>> UHD
>> >>>> develop version (master branch) I changed the UHD version to 3.10.2
>> >>>> (maint branch, #122bfae).
>> >>>>
>> >>>> To do so, I just used the prefix recipe gnuradio-stable via pybombs.
>> >>>>
>> >>>> Now I'm not able to transmit even low bandwidth signals. My setup is
>> >>>>
>> >>>> - X300 via 1G ethernet
>> >>>> - gnuradio (maint branch)
>> >>>> - Debian with kernel 4.9.
>> >>>>
>> >>>> I used a simple flow graph with just a sine source directly connected
>> >>>> to the USRP sink block. Even for sample rates below 2MSamples/s one
>> of
>> >>>> the cpu core of my Xeon E5-2630 v2 is 100% busy.
>> >>>>
>> >>>> Observing the cpu load with htop, I recognized that most of the cpu
>> >>>> load is red which indicates kernel space activities...
>> >>>>
>> >>>> After several seconds I finally got many "U"s in the console window.
>> >>>>
>> >>>> Some ideas how to isolate the problem?
>> >>>> Thanks in advance!
>> >>>>
>> >>>> Kai
>> >>>>
>> >>>> _______________________________________________
>> >>>> USRP-users mailing list
>> >>>> USRP-users at lists.ettus.com
>> >>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
>> >>>
>> >>> I just constructed the attached flow-graph, and ran it on my ancient
>> Dell
>> >>> Latitude laptop:
>> >>>
>> >>> [mleech at lab ~]$ ./x300_test.py
>> >>> linux; GNU C++ version 4.8.3 20140911 (Red Hat 4.8.3-7); Boost_105400;
>> >>> UHD_003.010.002.000-3-g122bfae1
>> >>>
>> >>> -- X300 initialization sequence...
>> >>> -- Determining maximum frame size... 4320 bytes.
>> >>> -- Setup basic communication...
>> >>> -- Loading values from EEPROM...
>> >>> -- Setup RF frontend clocking...
>> >>> -- Radio 1x clock:200
>> >>> -- Detecting internal GPSDO.... Found an internal GPSDO: LC_XO,
>> Firmware
>> >>> Rev
>> >>> 0.929a
>> >>> -- [DMA FIFO] Running BIST for FIFO 0... pass (Throughput: 1290.6MB/s)
>> >>> -- [DMA FIFO] Running BIST for FIFO 1... pass (Throughput: 1309.7MB/s)
>> >>> -- [RFNoC Radio] Performing register loopback test... pass
>> >>> -- [RFNoC Radio] Performing register loopback test... pass
>> >>> -- [RFNoC Radio] Performing register loopback test... pass
>> >>> -- [RFNoC Radio] Performing register loopback test... pass
>> >>> -- Performing timer loopback test... pass
>> >>> -- Performing timer loopback test... pass
>> >>>
>> >>>
>> >>> No underruns at all, and CPU was quite modest.
>> >>>
>> >>>
>> >>> Are you running within a VM?
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> _______________________________________________
>> >>> USRP-users mailing list
>> >>> USRP-users at lists.ettus.com
>> >>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
>> >>>
>> >>
>> >>
>> >
>> >
>> > _______________________________________________
>> > USRP-users mailing list
>> > USRP-users at lists.ettus.com
>> > http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
>>
>>
>>
>> --
>> Kai-Uwe Storek
>>
>>
>>
>> Privatsphäre auch bei eMails? eMail-Verschlüsselung leicht gemacht:
>>
>> Thunderbird: http://www.thunderbird-mail.de/wiki/Enigmail_OpenPGP
>> Outlook:https://code.google.com/p/outlook-privacy-plugin/
>> Outlook (kostenpflichtig): http://www.gpg4o.de/en/product
>> /productinfo-gpg4o.html
>>
>> Mein Schlüssel: http://goo.gl/QGVvsw
>>
>> _______________________________________________
>> USRP-users mailing list
>> USRP-users at lists.ettus.com
>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/attachments/20171024/8d2df843/attachment-0002.html>


More information about the USRP-users mailing list