usrp-users@lists.ettus.com

Discussion and technical support related to USRP, UHD, RFNoC

View all threads

x410 x4_200 connectivity question

TB
Tillson, Bob (US)
Wed, Jun 15, 2022 3:17 PM

So I am on the x4_200 image and am wondering what the "correct" way to setup the transport is.

I currently have a single qsfp28 connection on the x410 to a single 100 GbE port on a mellanox card.

Question #1:
Looking at the description of the fpga image it describes the x4_200 as "4 x 10GbE", so should I really be configured as single qsfp28 connection on the x410 to a quad 10 GbE on host with a breakout cable?

I am seeing some wonkiness in my app, which was pretty stable on the N310 and wondering if it could be due to my transport setup.
I am still investigating the app, but it made me think about how the data is delivered from device to host.

Question #2:
With the x4_200 fpga image, is the traffic for each RF port "routed" to a single 10GbE interface?  4 RF ports, 4 10GbE connections?

Any insights would be greatly appreciated :)

So I am on the x4_200 image and am wondering what the "correct" way to setup the transport is. I currently have a single qsfp28 connection on the x410 to a single 100 GbE port on a mellanox card. Question #1: Looking at the description of the fpga image it describes the x4_200 as "4 x 10GbE", so should I really be configured as single qsfp28 connection on the x410 to a quad 10 GbE on host with a breakout cable? I am seeing some wonkiness in my app, which was pretty stable on the N310 and wondering if it could be due to my transport setup. I am still investigating the app, but it made me think about how the data is delivered from device to host. Question #2: With the x4_200 fpga image, is the traffic for each RF port "routed" to a single 10GbE interface? 4 RF ports, 4 10GbE connections? Any insights would be greatly appreciated :)
WF
Wade Fife
Wed, Jun 15, 2022 8:35 PM

Hi Bob,

Question 1:
With the x4 bitstream, the X410 puts a separate 10 GbE interface on each
lane of its QSFP28 port 0 (the QSFP port has four lanes). There are
breakout cables that go from one QSFP to four SFPs if you have cards with
SFP ports, or, if your card supports four 10 GbE ports on a single QSFP
plug then you can use a single QSFP cable like you described. But not all
cards support this. Check your card's documentation.

Question 2:
This is configurable. You can use the addr, second_addr, third_addr, and
fourth_addr device arguments to indicate which IP ports to use for
streaming. UHD automatically distributes the streams over the ports you
indicate.

Thanks,

Wade

On Wed, Jun 15, 2022 at 10:19 AM Tillson, Bob (US) via USRP-users <
usrp-users@lists.ettus.com> wrote:

So I am on the x4_200 image and am wondering what the “correct” way to
setup the transport is.

I currently have a single qsfp28 connection on the x410 to a single 100
GbE port on a mellanox card.

Question #1:

Looking at the description of the fpga image it describes the x4_200 as “4
x 10GbE”, so should I really be configured as single qsfp28 connection on
the x410 to a quad 10 GbE on host with a breakout cable?

I am seeing some wonkiness in my app, which was pretty stable on the N310
and wondering if it could be due to my transport setup.

I am still investigating the app, but it made me think about how the data
is delivered from device to host.

Question #2:

With the x4_200 fpga image, is the traffic for each RF port “routed” to a
single 10GbE interface?  4 RF ports, 4 10GbE connections?

Any insights would be greatly appreciated J


USRP-users mailing list -- usrp-users@lists.ettus.com
To unsubscribe send an email to usrp-users-leave@lists.ettus.com

Hi Bob, Question 1: With the x4 bitstream, the X410 puts a separate 10 GbE interface on each lane of its QSFP28 port 0 (the QSFP port has four lanes). There are breakout cables that go from one QSFP to four SFPs if you have cards with SFP ports, or, if your card supports four 10 GbE ports on a single QSFP plug then you can use a single QSFP cable like you described. But not all cards support this. Check your card's documentation. Question 2: This is configurable. You can use the addr, second_addr, third_addr, and fourth_addr device arguments to indicate which IP ports to use for streaming. UHD automatically distributes the streams over the ports you indicate. Thanks, Wade On Wed, Jun 15, 2022 at 10:19 AM Tillson, Bob (US) via USRP-users < usrp-users@lists.ettus.com> wrote: > So I am on the x4_200 image and am wondering what the “correct” way to > setup the transport is. > > > > I currently have a single qsfp28 connection on the x410 to a single 100 > GbE port on a mellanox card. > > > > Question #1: > > Looking at the description of the fpga image it describes the x4_200 as “4 > x 10GbE”, so should I really be configured as single qsfp28 connection on > the x410 to a quad 10 GbE on host with a breakout cable? > > > > I am seeing some wonkiness in my app, which was pretty stable on the N310 > and wondering if it could be due to my transport setup. > > I am still investigating the app, but it made me think about how the data > is delivered from device to host. > > > > Question #2: > > With the x4_200 fpga image, is the traffic for each RF port “routed” to a > single 10GbE interface? 4 RF ports, 4 10GbE connections? > > > > Any insights would be greatly appreciated J > _______________________________________________ > USRP-users mailing list -- usrp-users@lists.ettus.com > To unsubscribe send an email to usrp-users-leave@lists.ettus.com >
PK
Piotr Krysik
Sun, Oct 23, 2022 7:27 PM

Hello Bob,

Were you able to use more than one 10 GbE connection to transport
traffic that exceeds throughput of single SFP+ 10GbE interface?

I wasn't able to do that yet. This is the command that I execute to test it:

prefix/lib/uhd/examples/benchmark_rate  --rx_subdev "A:0 B:0"
--rx_channels 0,1
--args="addr=192.168.10.2,second_addr=192.168.11.2,mgmt_addr=192.168.0.9"
--rx_rate 245.76e6 --rx_delay 4 --duration 10 --pps internal
--multi_streamer

In wireshark on the second interface I can see only UDP packets testing
MTU size and 3 first RFNoC management packets. And that's all. Only
first interface receives samples.

Best Regards,
Piotr Krysik

W dniu 15.06.2022 o 17:17, Tillson, Bob (US) via USRP-users pisze:

So I am on the x4_200 image and am wondering what the “correct” way to
setup the transport is.

I currently have a single qsfp28 connection on the x410 to a single
100 GbE port on a mellanox card.

Question #1:

Looking at the description of the fpga image it describes the x4_200
as “4 x 10GbE”, so should I really be configured as single qsfp28
connection on the x410 to a quad 10 GbE on host with a breakout cable?

I am seeing some wonkiness in my app, which was pretty stable on the
N310 and wondering if it could be due to my transport setup.

I am still investigating the app, but it made me think about how the
data is delivered from device to host.

Question #2:

With the x4_200 fpga image, is the traffic for each RF port “routed”
to a single 10GbE interface?  4 RF ports, 4 10GbE connections?

Any insights would be greatly appreciated J

Hello Bob, Were you able to use more than one 10 GbE connection to transport traffic that exceeds throughput of single SFP+ 10GbE interface? I wasn't able to do that yet. This is the command that I execute to test it: prefix/lib/uhd/examples/benchmark_rate  --rx_subdev "A:0 B:0" --rx_channels 0,1 --args="addr=192.168.10.2,second_addr=192.168.11.2,mgmt_addr=192.168.0.9" --rx_rate 245.76e6 --rx_delay 4 --duration 10 --pps internal --multi_streamer In wireshark on the second interface I can see only UDP packets testing MTU size and 3 first RFNoC management packets. And that's all. Only first interface receives samples. Best Regards, Piotr Krysik W dniu 15.06.2022 o 17:17, Tillson, Bob (US) via USRP-users pisze: > > So I am on the x4_200 image and am wondering what the “correct” way to > setup the transport is. > > I currently have a single qsfp28 connection on the x410 to a single > 100 GbE port on a mellanox card. > > Question #1: > > Looking at the description of the fpga image it describes the x4_200 > as “4 x 10GbE”, so should I really be configured as single qsfp28 > connection on the x410 to a quad 10 GbE on host with a breakout cable? > > I am seeing some wonkiness in my app, which was pretty stable on the > N310 and wondering if it could be due to my transport setup. > > I am still investigating the app, but it made me think about how the > data is delivered from device to host. > > Question #2: > > With the x4_200 fpga image, is the traffic for each RF port “routed” > to a single 10GbE interface?  4 RF ports, 4 10GbE connections? > > Any insights would be greatly appreciated J >
S
shikunhong12138@outlook.com
Thu, Apr 6, 2023 1:29 PM

For question 2, can I understand it like this: if I use mellanox mcx516ccat board, mirror select x4 200 for firmware, parameters addr, second_addr, third_addr and

The fourth_addr is set to 192.168.10.2, so that when uhd is running, the data stream will be automatically allocated, up to 40 GbE?

For question 2, can I understand it like this: if I use mellanox mcx516ccat board, mirror select x4 200 for firmware, parameters addr, second_addr, third_addr and The fourth_addr is set to 192.168.10.2, so that when uhd is running, the data stream will be automatically allocated, up to 40 GbE?
S
shikunhong12138@outlook.com
Thu, Apr 6, 2023 1:29 PM

For question 2, can I understand it like this: if I use mellanox mcx516ccat board, mirror select x4 200 for firmware, parameters addr, second_addr, third_addr and

The fourth_addr is set to 192.168.10.2, so that when uhd is running, the data stream will be automatically allocated, up to 40 GbE?

For question 2, can I understand it like this: if I use mellanox mcx516ccat board, mirror select x4 200 for firmware, parameters addr, second_addr, third_addr and The fourth_addr is set to 192.168.10.2, so that when uhd is running, the data stream will be automatically allocated, up to 40 GbE?
WF
Wade Fife
Thu, Apr 6, 2023 5:43 PM

I don't think that card supports 4 x 10 GbE. But suppose you used a card
that does, or you used multiple 10 GbE cards to connect to all four 10 GbE
ports on the X410, then you would set the args to
"addr=192.168.10.2,second_addr=192.168.11.2,third_addr=192.168.12.2,fourth_addr=192.168.13.2".
The streams would be automatically distributed across the four ethernet
devices corresponding to those IP address subnets, giving you a total
theoretical throughput of 40 Gbit/s.

Wade

On Thu, Apr 6, 2023 at 8:32 AM shikunhong12138@outlook.com wrote:

For question 2, can I understand it like this: if I use mellanox
mcx516ccat board, mirror select x4 200 for firmware, parameters addr,
second_addr, third_addr and

The fourth_addr is set to 192.168.10.2, so that when uhd is running, the
data stream will be automatically allocated, up to 40 GbE?


USRP-users mailing list -- usrp-users@lists.ettus.com
To unsubscribe send an email to usrp-users-leave@lists.ettus.com

I don't think that card supports 4 x 10 GbE. But suppose you used a card that does, or you used multiple 10 GbE cards to connect to all four 10 GbE ports on the X410, then you would set the args to "addr=192.168.10.2,second_addr=192.168.11.2,third_addr=192.168.12.2,fourth_addr=192.168.13.2". The streams would be automatically distributed across the four ethernet devices corresponding to those IP address subnets, giving you a total theoretical throughput of 40 Gbit/s. Wade On Thu, Apr 6, 2023 at 8:32 AM <shikunhong12138@outlook.com> wrote: > For question 2, can I understand it like this: if I use mellanox > mcx516ccat board, mirror select x4 200 for firmware, parameters addr, > second_addr, third_addr and > > The fourth_addr is set to 192.168.10.2, so that when uhd is running, the > data stream will be automatically allocated, up to 40 GbE? > _______________________________________________ > USRP-users mailing list -- usrp-users@lists.ettus.com > To unsubscribe send an email to usrp-users-leave@lists.ettus.com >
S
shikunhong12138@outlook.com
Fri, Apr 7, 2023 3:26 AM

Yes, you are right!

1. This should not support 4 10GbE, when I use CG 400 image, the ubuntu20 can automatically recognize it as 100 GbE

2. But there is a new problem: in gnuradio I can't change the sample rate, x410 devices always use a sample rate of 491.520 MHz, resulting in the computer can not output complete data to x410 devices in time.

Erro like this:

[INFO] [MPM.PeriphManager] init() called with device args `fpga=CG_400,mgmt_addr=192.168.10.2,name=ni-x4xx-324AA5D,product=x410,clock_source=internal,time_source=internal'.

usrp_source :warning: Requested sample rate 4.9152e+06 Hz not set; instead, 4.9152e+08 Hz used.

[WARNING] [0/Radio#1] Requesting invalid sampling rate from device: 4.9152 MHz. Actual rate is: 491.52 MHz.

[WARNING] [MULTI_USRP] Could not set RX rate to 4.915 MHz. Actual rate is 491.520 MHz

[WARNING] [0/Radio#0] Requesting invalid sampling rate from device: 4.9152 MHz. Actual rate is: 491.52 MHz.

[WARNING] [MULTI_USRP] Could not set TX rate to 4.915 MHz. Actual rate is 491.520 MHz

So I wondered how to change the sample rate of the device under the CG 400 image?

Any insights would be greatly appreciated:)

Yes, you are right! 1\. This should not support 4 10GbE, when I use CG 400 image, the ubuntu20 can automatically recognize it as 100 GbE 2\. But there is a new problem: in gnuradio I can't change the sample rate, x410 devices always use a sample rate of 491.520 MHz, resulting in the computer can not output complete data to x410 devices in time. Erro like this: **\[INFO\] \[MPM.PeriphManager\] init() called with device args \`fpga=CG_400,mgmt_addr=192.168.10.2,name=ni-x4xx-324AA5D,product=x410,clock_source=internal,time_source=internal'.** **usrp_source :warning: Requested sample rate 4.9152e+06 Hz not set; instead, 4.9152e+08 Hz used.** **\[WARNING\] \[0/Radio#1\] Requesting invalid sampling rate from device: 4.9152 MHz. Actual rate is: 491.52 MHz.** **\[WARNING\] \[MULTI_USRP\] Could not set RX rate to 4.915 MHz. Actual rate is 491.520 MHz** **\[WARNING\] \[0/Radio#0\] Requesting invalid sampling rate from device: 4.9152 MHz. Actual rate is: 491.52 MHz.** **\[WARNING\] \[MULTI_USRP\] Could not set TX rate to 4.915 MHz. Actual rate is 491.520 MHz** So I wondered how to change the sample rate of the device under the CG 400 image? Any insights would be greatly appreciated:)
H
h57jafari@gmail.com
Fri, Apr 7, 2023 3:10 PM
  • Utilizing the CG_400 bitfile which does not include DUC/DDC resampling thus supports only rates of 491.52 MS/s and 500 MS/s.
* Utilizing the CG_400 bitfile which does not include DUC/DDC resampling thus supports only rates of 491.52 MS/s and 500 MS/s.