usrp-users@lists.ettus.com

Discussion and technical support related to USRP, UHD, RFNoC

View all threads

N300 test with 10G port

TL
Trung Le Nguyen Anh
Fri, Nov 18, 2022 5:45 PM

Hi,

Currently, I connect my PC to N300 via X520 DA1 card with a SFP+ fiber adapter. Both ends shows speed 10000Mbps.
But when I use Iperf3 to check, the bandwidth is just 173Mb/s.

On PC:
sudo ethtool enp3s0
[sudo] password for tma:
Settings for enp3s0:
Supported ports: [ FIBRE ]
Supported link modes:  10000baseT/Full
Supported pause frame use: Symmetric
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes:  10000baseT/Full
Advertised pause frame use: Symmetric
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Port: FIBRE
PHYAD: 0
Transceiver: internal
Auto-negotiation: off
Supports Wake-on: umbg
Wake-on: g
Current message level: 0x00000007 (7)
drv probe link
Link detected: yes

On N300:
root@ni-n3xx-325A9D8:~# ethtool sfp1
Settings for sfp1:
Supported ports: [ ]
Supported link modes:  10000baseER/Full
Supported pause frame use: Symmetric Receive-only
Supports auto-negotiation: No
Supported FEC modes: Not reported
Advertised link modes:  10000baseER/Full
Advertised pause frame use: Symmetric Receive-only
Advertised auto-negotiation: No
Advertised FEC modes: Not reported
Speed: 10000Mb/s
Duplex: Full
Port: MII
PHYAD: 4
Transceiver: internal
Auto-negotiation: off
Link detected: yes

I tested with benchmark with command below. I don't know if this result is good to run with gNB?

/usr/local/lib/uhd/examples/benchmark_rate    --args "type=n3xx,mgmt_addr=192.168.30.3,addr=192.168.20.2,master_clock_rate=125e6"    --duration 60    --channels "0"    --rx_rate 122.07e6    --rx_subdev "A:0"    --tx_rate 122.07e6    --tx_subdev "A:0"

[INFO] [UHD] linux; GNU C++ version 7.5.0; Boost_106501; UHD_4.2.0.HEAD-0-g46a70d85
[00:00:00.000140] Creating the usrp device with: type=n3xx,mgmt_addr=192.168.30.3,addr=192.168.20.2,master_clock_rate=125e6...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=192.168.30.3,type=n3xx,product=n300,serial=325A9D8,name=ni-n3xx-325A9D8,fpga=HG,claimed=False,addr=192.168.20.2,master_clock_rate=125e6
[INFO] [MPM.PeriphManager] init() called with device args `fpga=HG,master_clock_rate=125e6,mgmt_addr=192.168.30.3,name=ni-n3xx-325A9D8,product=n300,clock_source=internal,time_source=internal'.
Using Device: Single USRP:
Device: N300-Series Device
Mboard 0: n300
RX Channel: 0
RX DSP: 0
RX Dboard: A
RX Subdev: Magnesium
TX Channel: 0
TX DSP: 0
TX Dboard: A
TX Subdev: Magnesium

[00:00:02.680810005] Setting device timestamp to 0...
[WARNING] [MULTI_USRP] Could not set RX rate to 122.070 MHz. Actual rate is 125.000 MHz
[WARNING] [MULTI_USRP] Could not set TX rate to 122.070 MHz. Actual rate is 125.000 MHz
[00:00:02.683219534] Testing receive rate 125.000000 Msps on 1 channels
Setting TX spp to 2000
[00:00:02.684711859] Testing transmit rate 125.000000 Msps on 1 channels
UU[00:01:02.686170124] Benchmark complete.

Benchmark rate summary:
Num received samples:    7499635293
Num dropped samples:      0
Num overruns detected:    0
Num transmitted samples:  7499530000
Num sequence errors (Tx): 0
Num sequence errors (Rx): 0
Num underruns detected:  2
Num late commands:        0
Num timeouts (Tx):        0
Num timeouts (Rx):        0

Done!

Thanks,
Trung.

Hi, Currently, I connect my PC to N300 via X520 DA1 card with a SFP+ fiber adapter. Both ends shows speed 10000Mbps. But when I use Iperf3 to check, the bandwidth is just 173Mb/s. On PC: sudo ethtool enp3s0 [sudo] password for tma: Settings for enp3s0: Supported ports: [ FIBRE ] Supported link modes: 10000baseT/Full Supported pause frame use: Symmetric Supports auto-negotiation: No Supported FEC modes: Not reported Advertised link modes: 10000baseT/Full Advertised pause frame use: Symmetric Advertised auto-negotiation: No Advertised FEC modes: Not reported Speed: 10000Mb/s Duplex: Full Port: FIBRE PHYAD: 0 Transceiver: internal Auto-negotiation: off Supports Wake-on: umbg Wake-on: g Current message level: 0x00000007 (7) drv probe link Link detected: yes On N300: root@ni-n3xx-325A9D8:~# ethtool sfp1 Settings for sfp1: Supported ports: [ ] Supported link modes: 10000baseER/Full Supported pause frame use: Symmetric Receive-only Supports auto-negotiation: No Supported FEC modes: Not reported Advertised link modes: 10000baseER/Full Advertised pause frame use: Symmetric Receive-only Advertised auto-negotiation: No Advertised FEC modes: Not reported Speed: 10000Mb/s Duplex: Full Port: MII PHYAD: 4 Transceiver: internal Auto-negotiation: off Link detected: yes I tested with benchmark with command below. I don't know if this result is good to run with gNB? /usr/local/lib/uhd/examples/benchmark_rate --args "type=n3xx,mgmt_addr=192.168.30.3,addr=192.168.20.2,master_clock_rate=125e6" --duration 60 --channels "0" --rx_rate 122.07e6 --rx_subdev "A:0" --tx_rate 122.07e6 --tx_subdev "A:0" [INFO] [UHD] linux; GNU C++ version 7.5.0; Boost_106501; UHD_4.2.0.HEAD-0-g46a70d85 [00:00:00.000140] Creating the usrp device with: type=n3xx,mgmt_addr=192.168.30.3,addr=192.168.20.2,master_clock_rate=125e6... [INFO] [MPMD] Initializing 1 device(s) in parallel with args: mgmt_addr=192.168.30.3,type=n3xx,product=n300,serial=325A9D8,name=ni-n3xx-325A9D8,fpga=HG,claimed=False,addr=192.168.20.2,master_clock_rate=125e6 [INFO] [MPM.PeriphManager] init() called with device args `fpga=HG,master_clock_rate=125e6,mgmt_addr=192.168.30.3,name=ni-n3xx-325A9D8,product=n300,clock_source=internal,time_source=internal'. Using Device: Single USRP: Device: N300-Series Device Mboard 0: n300 RX Channel: 0 RX DSP: 0 RX Dboard: A RX Subdev: Magnesium TX Channel: 0 TX DSP: 0 TX Dboard: A TX Subdev: Magnesium [00:00:02.680810005] Setting device timestamp to 0... [WARNING] [MULTI_USRP] Could not set RX rate to 122.070 MHz. Actual rate is 125.000 MHz [WARNING] [MULTI_USRP] Could not set TX rate to 122.070 MHz. Actual rate is 125.000 MHz [00:00:02.683219534] Testing receive rate 125.000000 Msps on 1 channels Setting TX spp to 2000 [00:00:02.684711859] Testing transmit rate 125.000000 Msps on 1 channels UU[00:01:02.686170124] Benchmark complete. Benchmark rate summary: Num received samples: 7499635293 Num dropped samples: 0 Num overruns detected: 0 Num transmitted samples: 7499530000 Num sequence errors (Tx): 0 Num sequence errors (Rx): 0 Num underruns detected: 2 Num late commands: 0 Num timeouts (Tx): 0 Num timeouts (Rx): 0 Done! Thanks, Trung.
MD
Marcus D. Leech
Fri, Nov 18, 2022 7:23 PM

On 18/11/2022 12:45, Trung Le Nguyen Anh wrote:

Hi,

Currently, I connect my PC to N300 via X520 DA1 card with a SFP+ fiber
adapter. Both ends shows speed 10000Mbps.
But when I use Iperf3 to check, the bandwidth is just 173Mb/s.

On PC:
sudo ethtool enp3s0
[sudo] password for tma:
Settings for enp3s0:
        Supported ports: [ FIBRE ]
        Supported link modes: 10000baseT/Full
        Supported pause frame use: Symmetric
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  10000baseT/Full
        Advertised pause frame use: Symmetric
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: 10000Mb/s
        Duplex: Full
        Port: FIBRE
        PHYAD: 0
        Transceiver: internal
        Auto-negotiation: off
        Supports Wake-on: umbg
        Wake-on: g
        Current message level: 0x00000007 (7)
                               drv probe link
        Link detected: yes

On N300:
root@ni-n3xx-325A9D8:~# ethtool sfp1
Settings for sfp1:
        Supported ports: [ ]
        Supported link modes: 10000baseER/Full
        Supported pause frame use: Symmetric Receive-only
        Supports auto-negotiation: No
        Supported FEC modes: Not reported
        Advertised link modes:  10000baseER/Full
        Advertised pause frame use: Symmetric Receive-only
        Advertised auto-negotiation: No
        Advertised FEC modes: Not reported
        Speed: 10000Mb/s
        Duplex: Full
        Port: MII
        PHYAD: 4
        Transceiver: internal
        Auto-negotiation: off
        Link detected: yes

I tested with benchmark with command below. I don't know if this
result is good to run with gNB?

/usr/local/lib/uhd/examples/benchmark_rate     --args
"type=n3xx,mgmt_addr=192.168.30.3,addr=192.168.20.2,master_clock_rate=125e6"
   --duration 60    --channels "0"    --rx_rate 122.07e6  --rx_subdev
"A:0"    --tx_rate 122.07e6    --tx_subdev "A:0"

[INFO] [UHD] linux; GNU C++ version 7.5.0; Boost_106501;
UHD_4.2.0.HEAD-0-g46a70d85
[00:00:00.000140] Creating the usrp device with:
type=n3xx,mgmt_addr=192.168.30.3,addr=192.168.20.2,master_clock_rate=125e6...
[INFO] [MPMD] Initializing 1 device(s) in parallel with args:
mgmt_addr=192.168.30.3,type=n3xx,product=n300,serial=325A9D8,name=ni-n3xx-325A9D8,fpga=HG,claimed=False,addr=192.168.20.2,master_clock_rate=125e6
[INFO] [MPM.PeriphManager] init() called with device args
`fpga=HG,master_clock_rate=125e6,mgmt_addr=192.168.30.3,name=ni-n3xx-325A9D8,product=n300,clock_source=internal,time_source=internal'.
Using Device: Single USRP:
  Device: N300-Series Device
  Mboard 0: n300
  RX Channel: 0
    RX DSP: 0
    RX Dboard: A
    RX Subdev: Magnesium
  TX Channel: 0
    TX DSP: 0
    TX Dboard: A
    TX Subdev: Magnesium

[00:00:02.680810005] Setting device timestamp to 0...
[WARNING] [MULTI_USRP] Could not set RX rate to 122.070 MHz. Actual
rate is 125.000 MHz
[WARNING] [MULTI_USRP] Could not set TX rate to 122.070 MHz. Actual
rate is 125.000 MHz
[00:00:02.683219534] Testing receive rate 125.000000 Msps on 1 channels
Setting TX spp to 2000
[00:00:02.684711859] Testing transmit rate 125.000000 Msps on 1 channels
UU[00:01:02.686170124] Benchmark complete.

Benchmark rate summary:
  Num received samples: 7499635293
  Num dropped samples:      0
  Num overruns detected:    0
  Num transmitted samples:  7499530000
  Num sequence errors (Tx): 0
  Num sequence errors (Rx): 0
  Num underruns detected:   2
  Num late commands:        0
  Num timeouts (Tx):        0
  Num timeouts (Rx):        0

Done!

Thanks,
Trung.


USRP-users mailing list --usrp-users@lists.ettus.com
To unsubscribe send an email tousrp-users-leave@lists.ettus.com

My guess is that iPerf uses ICMP packets that are terminated on the CPU
of the N300, and not in the (fast) FPGA fabric.

Your benchmark_rate test indicates it can support 125Msps to your host
computer--that's the critical test.

On 18/11/2022 12:45, Trung Le Nguyen Anh wrote: > Hi, > > Currently, I connect my PC to N300 via X520 DA1 card with a SFP+ fiber > adapter. Both ends shows speed 10000Mbps. > But when I use Iperf3 to check, the bandwidth is just 173Mb/s. > > On PC: > sudo ethtool enp3s0 > [sudo] password for tma: > Settings for enp3s0: >         Supported ports: [ FIBRE ] >         Supported link modes: 10000baseT/Full >         Supported pause frame use: Symmetric >         Supports auto-negotiation: No >         Supported FEC modes: Not reported >         Advertised link modes:  10000baseT/Full >         Advertised pause frame use: Symmetric >         Advertised auto-negotiation: No >         Advertised FEC modes: Not reported >         Speed: 10000Mb/s >         Duplex: Full >         Port: FIBRE >         PHYAD: 0 >         Transceiver: internal >         Auto-negotiation: off >         Supports Wake-on: umbg >         Wake-on: g >         Current message level: 0x00000007 (7) >                                drv probe link >         Link detected: yes > > On N300: > root@ni-n3xx-325A9D8:~# ethtool sfp1 > Settings for sfp1: >         Supported ports: [ ] >         Supported link modes: 10000baseER/Full >         Supported pause frame use: Symmetric Receive-only >         Supports auto-negotiation: No >         Supported FEC modes: Not reported >         Advertised link modes:  10000baseER/Full >         Advertised pause frame use: Symmetric Receive-only >         Advertised auto-negotiation: No >         Advertised FEC modes: Not reported >         Speed: 10000Mb/s >         Duplex: Full >         Port: MII >         PHYAD: 4 >         Transceiver: internal >         Auto-negotiation: off >         Link detected: yes > > I tested with benchmark with command below. I don't know if this > result is good to run with gNB? > > /usr/local/lib/uhd/examples/benchmark_rate     --args > "type=n3xx,mgmt_addr=192.168.30.3,addr=192.168.20.2,master_clock_rate=125e6" >    --duration 60    --channels "0"    --rx_rate 122.07e6  --rx_subdev > "A:0"    --tx_rate 122.07e6    --tx_subdev "A:0" > > [INFO] [UHD] linux; GNU C++ version 7.5.0; Boost_106501; > UHD_4.2.0.HEAD-0-g46a70d85 > [00:00:00.000140] Creating the usrp device with: > type=n3xx,mgmt_addr=192.168.30.3,addr=192.168.20.2,master_clock_rate=125e6... > [INFO] [MPMD] Initializing 1 device(s) in parallel with args: > mgmt_addr=192.168.30.3,type=n3xx,product=n300,serial=325A9D8,name=ni-n3xx-325A9D8,fpga=HG,claimed=False,addr=192.168.20.2,master_clock_rate=125e6 > [INFO] [MPM.PeriphManager] init() called with device args > `fpga=HG,master_clock_rate=125e6,mgmt_addr=192.168.30.3,name=ni-n3xx-325A9D8,product=n300,clock_source=internal,time_source=internal'. > Using Device: Single USRP: >   Device: N300-Series Device >   Mboard 0: n300 >   RX Channel: 0 >     RX DSP: 0 >     RX Dboard: A >     RX Subdev: Magnesium >   TX Channel: 0 >     TX DSP: 0 >     TX Dboard: A >     TX Subdev: Magnesium > > [00:00:02.680810005] Setting device timestamp to 0... > [WARNING] [MULTI_USRP] Could not set RX rate to 122.070 MHz. Actual > rate is 125.000 MHz > [WARNING] [MULTI_USRP] Could not set TX rate to 122.070 MHz. Actual > rate is 125.000 MHz > [00:00:02.683219534] Testing receive rate 125.000000 Msps on 1 channels > Setting TX spp to 2000 > [00:00:02.684711859] Testing transmit rate 125.000000 Msps on 1 channels > UU[00:01:02.686170124] Benchmark complete. > > > Benchmark rate summary: >   Num received samples: 7499635293 >   Num dropped samples:      0 >   Num overruns detected:    0 >   Num transmitted samples:  7499530000 >   Num sequence errors (Tx): 0 >   Num sequence errors (Rx): 0 >   Num underruns detected:   2 >   Num late commands:        0 >   Num timeouts (Tx):        0 >   Num timeouts (Rx):        0 > > > Done! > > Thanks, > Trung. > > _______________________________________________ > USRP-users mailing list --usrp-users@lists.ettus.com > To unsubscribe send an email tousrp-users-leave@lists.ettus.com My guess is that iPerf uses ICMP packets that are terminated on the CPU of the N300, and not in the (fast) FPGA fabric. Your benchmark_rate test indicates it can support 125Msps to your host computer--that's the critical test.