I recently had some real trouble with some blocks not being addressable and the trace log output of a probe telling me "Nothing connected on device:1/xbar:0->1. Ignoring that path." I need to minimize the size of the crossbar, and I only have static connections in my system so I was trying to build a minimal crossbar and apparently failing to understand the crossbar_routes table. Looking here: https://github.com/EttusResearch/uhd/blob/master/fpga/usrp3/top/x400/x440_CG_1600_rfnoc_image_core.yml#L40 Pasted here for completeness: # Rows correspond to input ports and columns correspond to output ports. # Entering a 1 includes and a 0 removes that route from the crossbar. crossbar_routes: # eth0 dma ep1 # eth1 ep0 - [ 1, 0, 0, 1, 1 ] # eth0 (QSFP Port 0) - [ 0, 1, 0, 1, 1 ] # eth1 (QSFP Port 1) - [ 0, 0, 1, 1, 1 ] # dma - [ 1, 1, 1, 0, 0 ] # ep0 (radio0.0) - [ 1, 1, 1, 0, 0 ] # ep1 (radio1.0) I am not sure input/output helps me here, but maybe from/to, so my first assumption is that I read the table such that across the top I say "From" and across the side I say "To". Is this correct or not? Next, since all the examples I see are symmetric in communication, I am not sure I fully understand any type of asymmetric connections. For example, my application is configured/monitored by localhost on the ARM on an X440, RX endpoints remote streams out 100Gbe/QSFP+ ports but doesn't need to receive anything from them, and TX endpoints receives from 100 Gbe/QSFP+. I'd also like my application to be able to send data out of the QSFP+ ports as well. I am not sure how to achieve this in the routes. I believe for connectivity I want to convey: *From* -> *To* dma -> eth0, eth1, txep eth0 -> txep eth1 -> txep txep -> dma rx0ep -> eth0 rx1ep -> eth1 # Block connections radio0 -> rx0ep radio1 -> rx1ep txep[3:0] -> radio0 txep[7:4] -> radio1 Note that txep has no streaming outputs, and rx0ep/rx1ep have no streaming inputs. Also note that my control port is on txep. I think this is why I need the dma connected to txep in both directions, right? Lastly, in the original table, eth0 was connected to eth0. Same with the other transport adapters (eth1 and dma) - is this actually necessary? Is a route built here, or is it ignored? Can these entries be set to 0? In terms of a crossbar, I would have thought the diagonal is all 0's since we're never going to be connecting to ourselves? To convey this, here is my table: crossbar_routes: # eth0 eth1 dma txep rx0ep, rx1ep - [ 0, 0, 1, 0, 1, 0] # eth0 (QSFP Port 0) - [ 0, 0, 1, 0, 0, 1] # eth1 (QSFP Port 1) - [ 0, 0, 0, 1, 0, 0] # dma - [ 1, 1, 1, 0, 0, 0] # txep - [ 0, 0, 0, 0, 0, 0] # rx0ep - [ 0, 0, 0, 0, 0, 0] # rx1ep Did I get it right, or am I completely wrong here? I've tried lots of combinations, but the only one that works is when I am basically symmetric and fully connected with all the TA's which is suboptimal. Any guidance would be much appreciated. Thanks, Brian

  //---------------------------------------------------------------------------
  // CHDR Crossbar
  //---------------------------------------------------------------------------

  wire [PORT_W-1:0] xb_to_ep0_tdata ;
  wire              xb_to_ep0_tlast ;
  wire              xb_to_ep0_tvalid;
  wire              xb_to_ep0_tready;
  wire [PORT_W-1:0] ep0_to_xb_tdata ;
  wire              ep0_to_xb_tlast ;
  wire              ep0_to_xb_tvalid;
  wire              ep0_to_xb_tready;
  wire [PORT_W-1:0] xb_to_ep1_tdata ;
  wire              xb_to_ep1_tlast ;
  wire              xb_to_ep1_tvalid;
  wire              xb_to_ep1_tready;
  wire [PORT_W-1:0] ep1_to_xb_tdata ;
  wire              ep1_to_xb_tlast ;
  wire              ep1_to_xb_tvalid;
  wire              ep1_to_xb_tready;
  wire [PORT_W-1:0] xb_to_ep2_tdata ;
  wire              xb_to_ep2_tlast ;
  wire              xb_to_ep2_tvalid;
  wire              xb_to_ep2_tready;
...

Hi Brian,\ \ I had to play around with the crossbar route matrix myself yet,\ but looking at the code it seems like [at least for transport adapters the routes to themselves have to be included for route discovery purposes](https://github.com/EttusResearch/uhd/blob/master/host/python/uhd/rfnoc_utils/builder_config.py#L346) from the UHD host.\ From the example in the image_core.yaml that you linked it seems like this does not have to be the case for SEPs.\ \ When the image builder is run, it then gets translated into HDL code via the [mako template for the rfnoc_image_core.sv](https://github.com/EttusResearch/uhd/blob/master/host/python/uhd/rfnoc_utils/templates/rfnoc_image_core.sv.mako#L111).\ \ You could try and run the `rfnoc_image_builder` in `--generate-only` mode with your settings and check what the generated rfnoc_image_core.sv looks like.\ \ Looking at a default x410_X4_200 I have built previously it seems that the default is fully symetric for SEP connections: > ``` > //--------------------------------------------------------------------------- > // CHDR Crossbar > //--------------------------------------------------------------------------- > > wire [PORT_W-1:0] xb_to_ep0_tdata ; > wire xb_to_ep0_tlast ; > wire xb_to_ep0_tvalid; > wire xb_to_ep0_tready; > wire [PORT_W-1:0] ep0_to_xb_tdata ; > wire ep0_to_xb_tlast ; > wire ep0_to_xb_tvalid; > wire ep0_to_xb_tready; > wire [PORT_W-1:0] xb_to_ep1_tdata ; > wire xb_to_ep1_tlast ; > wire xb_to_ep1_tvalid; > wire xb_to_ep1_tready; > wire [PORT_W-1:0] ep1_to_xb_tdata ; > wire ep1_to_xb_tlast ; > wire ep1_to_xb_tvalid; > wire ep1_to_xb_tready; > wire [PORT_W-1:0] xb_to_ep2_tdata ; > wire xb_to_ep2_tlast ; > wire xb_to_ep2_tvalid; > wire xb_to_ep2_tready; > ... > ``` \ Maybe someone else has used this feature more and can give more details.\ \ Regards,\ Niels --- \ Brian Padalino wrote: > I recently had some real trouble with some blocks not being addressable and > the trace log output of a probe telling me "Nothing connected on > device:1/xbar:0->1. Ignoring that path." > > I need to minimize the size of the crossbar, and I only have static > connections in my system so I was trying to build a minimal crossbar and > apparently failing to understand the crossbar_routes table. > > Looking here: > > https://github.com/EttusResearch/uhd/blob/master/fpga/usrp3/top/x400/x440_CG_1600_rfnoc_image_core.yml#L40 > > Pasted here for completeness: > > # Rows correspond to input ports and columns correspond to output ports. > > # Entering a 1 includes and a 0 removes that route from the crossbar. > > crossbar_routes: > > # eth0 dma ep1 > > # eth1 ep0 > > * \[ 1, 0, 0, 1, 1 \] # eth0 (QSFP Port 0) > * \[ 0, 1, 0, 1, 1 \] # eth1 (QSFP Port 1) > * \[ 0, 0, 1, 1, 1 \] # dma > * \[ 1, 1, 1, 0, 0 \] # ep0 (radio0.0) > * \[ 1, 1, 1, 0, 0 \] # ep1 (radio1.0) > > I am not sure input/output helps me here, but maybe from/to, so my first > assumption is that I read the table such that across the top I say "From" > and across the side I say "To". Is this correct or not? > > Next, since all the examples I see are symmetric in communication, I am not > sure I fully understand any type of asymmetric connections. For example, my > application is configured/monitored by localhost on the ARM on an X440, RX > endpoints remote streams out 100Gbe/QSFP+ ports but doesn't need to receive > anything from them, and TX endpoints receives from 100 Gbe/QSFP+. I'd also > like my application to be able to send data out of the QSFP+ ports as well. > > I am not sure how to achieve this in the routes. I believe for connectivity > I want to convey: > *From* -> *To* > dma -> eth0, eth1, txep > eth0 -> txep > eth1 -> txep > txep -> dma > rx0ep -> eth0 > rx1ep -> eth1 > > # Block connections > > radio0 -> rx0ep > radio1 -> rx1ep > txep\[3:0\] -> radio0 > txep\[7:4\] -> radio1 > > Note that txep has no streaming outputs, and rx0ep/rx1ep have no streaming > inputs. Also note that my control port is on txep. I think this is why I > need the dma connected to txep in both directions, right? > > Lastly, in the original table, eth0 was connected to eth0. Same with the > other transport adapters (eth1 and dma) - is this actually necessary? Is a > route built here, or is it ignored? Can these entries be set to 0? In terms > of a crossbar, I would have thought the diagonal is all 0's since we're > never going to be connecting to ourselves? > > To convey this, here is my table: > > crossbar_routes: > > # eth0 eth1 dma txep rx0ep, rx1ep > > * \[ 0, 0, 1, 0, 1, 0\] # eth0 (QSFP Port 0) > * \[ 0, 0, 1, 0, 0, 1\] # eth1 (QSFP Port 1) > * \[ 0, 0, 0, 1, 0, 0\] # dma > * \[ 1, 1, 1, 0, 0, 0\] # txep > * \[ 0, 0, 0, 0, 0, 0\] # rx0ep > * \[ 0, 0, 0, 0, 0, 0\] # rx1ep > > Did I get it right, or am I completely wrong here? I've tried lots of > combinations, but the only one that works is when I am basically symmetric > and fully connected with all the TA's which is suboptimal. > > Any guidance would be much appreciated. > > Thanks, > Brian