I'm having a bit of a hard time understanding the minimal requirements for the CHDR Crossbar and connectivity. I'm working with an X440, so I have 3 transport adapters (int0, sfp0, sfp1), 2 blocks (radio0, radio1), 2 TX endpoints each with 4 ports (tx0, tx1), and 8 RX endpoints each with 1 port (rx[0-7]). I have tx0 ctrlport enabled, and none of the other ctrlports are enabled. I know I want rx[0-3] to only ever stream out of sfp0, and I want rx[4-7] to only ever stream out of sfp1. I want tx0 and tx1 to both receive CHDR packets from sfp0 and sfp1. I will always configure the device via int0. I also notice that rfnoc_core_kernel has a parameter for CHDR_XBAR_PRESENT. The comment for the parameter states: "1 if the CHDR crossbar is present. If 0 then transports are directly connected to SEPs". Connecting everything through the crossbar even with a sparse routing matrix ends up with around 18kLUT utilization. Since I know I want this extremely fixed and rigid design, I've got some questions: - How much of the CHDR crossbar can I remove? Can I get rid of it altogether? Are there any examples of a design with no CHDR crossbar? - Can I combine the RX SEPs into a single port per SFP connection using an AXI-Streaming mux of some type? Or is this accomplished in the same way in the crossbar with a sparse routing matrix? - How would one connect the multiple SEPs directly to the TA without going through the crossbar as the CHDR_XBAR_PRESENT parameter suggests is possible? Is it possible to describe this in the yaml file or does it require hand editing the generated rfnoc_image_core.sv file? - Since configuration is happening from int0, and tx0 is the only SEP with a ctrlport on it, does this suggest I need int0 to only be connected to tx0 in the connections and it doesn't need to go anywhere else? I will note that I tried this and I received a message saying a route couldn't be found for my remote streams. Is this maybe an oversight with remote streaming and sparse connectivity in the crossbar? I appreciate any insights you might be able to give. Thanks, Brian

- The only way to combine multiple SEPs into one is if you mux/demux in software as well as in a block after the SEP. The virtual channel feature was so supposed to address this, but as you know, we never implemented it. - I think if you removed the crossbar, UHD/RFNoC wouldn't balk at first, since it does a dynamic topology discovery, but I'm still not sure everything would work. We had originally thought we would support any number of crossbars (0, 1, 2, ...) but then eventually decided we'll use the routing matrix feature instead, and always assume a crossbar. - This means you cannot use the rfnoc_image_builder workflow to remove the crossbar. You would need to generate your files with rfnoc_image_builder -G, then manually remove the crossbar, then call make directly or do rfnoc_image_builder --reuse. - Like I said, not sure if this works at all. But it will definitely not work if you don't observe these things: - The primary connection to the device (from UHDs perspective) must have access to an SEP with a ctrl interface. We also have a known issue where the first SEP that UHD connects to needs to be the one with the ctrl interface. - I'm not sure about the route-not-found error. Like I said, you're doing something way out of spec. --M On Wed, Jul 9, 2025 at 6:32 PM Brian Padalino <bpadalino@gmail.com> wrote: > I'm having a bit of a hard time understanding the minimal requirements for > the CHDR Crossbar and connectivity. > > I'm working with an X440, so I have 3 transport adapters (int0, sfp0, > sfp1), 2 blocks (radio0, radio1), 2 TX endpoints each with 4 ports (tx0, > tx1), and 8 RX endpoints each with 1 port (rx[0-7]). > > I have tx0 ctrlport enabled, and none of the other ctrlports are enabled. > I know I want rx[0-3] to only ever stream out of sfp0, and I want rx[4-7] > to only ever stream out of sfp1. I want tx0 and tx1 to both receive CHDR > packets from sfp0 and sfp1. I will always configure the device via int0. > > I also notice that rfnoc_core_kernel has a parameter for > CHDR_XBAR_PRESENT. The comment for the parameter states: "1 if the CHDR > crossbar is present. If 0 then transports are directly connected to SEPs". > > Connecting everything through the crossbar even with a sparse routing > matrix ends up with around 18kLUT utilization. > > Since I know I want this extremely fixed and rigid design, I've got some > questions: > > - How much of the CHDR crossbar can I remove? Can I get rid of it > altogether? Are there any examples of a design with no CHDR crossbar? > > - Can I combine the RX SEPs into a single port per SFP connection using > an AXI-Streaming mux of some type? Or is this accomplished in the same way > in the crossbar with a sparse routing matrix? > > - How would one connect the multiple SEPs directly to the TA without > going through the crossbar as the CHDR_XBAR_PRESENT parameter suggests is > possible? Is it possible to describe this in the yaml file or does it > require hand editing the generated rfnoc_image_core.sv file? > > - Since configuration is happening from int0, and tx0 is the only SEP > with a ctrlport on it, does this suggest I need int0 to only be connected > to tx0 in the connections and it doesn't need to go anywhere else? I will > note that I tried this and I received a message saying a route couldn't be > found for my remote streams. Is this maybe an oversight with remote > streaming and sparse connectivity in the crossbar? > > I appreciate any insights you might be able to give. > > Thanks, > Brian > _______________________________________________ > USRP-users mailing list -- usrp-users@lists.ettus.com > To unsubscribe send an email to usrp-users-leave@lists.ettus.com >

Hey Martin, Thanks for the clarification. I understand I'm pushing a lot of boundaries and I appreciate the time you're putting in to explain. Going back to the route-not-found error, this configuration is not necessarily out of spec, I feel. I probably wasn't clear with the intention there. For the current state of how I have things working: - The arm runs a python script to initialize the RFNoC graph and start streams. This all goes through int0. - SEP tx0 has a control port and is able to communicate the configurations and is the first SEP in the list - I have 8 RX SEP rx0-7, where rx0-3 have a connection in the crossbar to sfp0 and rx4-7 have a connection in the crossbar to sfp1 - All 8 RX SEP have a connection to and from int0 - All SEPs are being set to remotely stream My issue arises when I remove the RX SEP connections to/from int0 - I get that route-not-found error. My inclination is that since the RX SEPs only have an output data port and no ctrl port, it should be fine to only connect them in the crossbar to the TA they need to send their data. Do you think it's possible there is a bug in UHD when validating the destination is reachable it is not considering a remote streaming connection and is trying to connect back up to the SW endpoint? Thanks, Brian On Thu, Jul 10, 2025 at 6:19 AM Martin Braun <martin.braun@ettus.com> wrote: > - The only way to combine multiple SEPs into one is if you mux/demux in > software as well as in a block after the SEP. The virtual channel > feature was so supposed to address this, but as you know, we never > implemented it. > - I think if you removed the crossbar, UHD/RFNoC wouldn't balk at first, > since it does a dynamic topology discovery, but I'm still not sure > everything would work. We had originally thought we would support any > number of crossbars (0, 1, 2, ...) but then eventually decided we'll use > the routing matrix feature instead, and always assume a crossbar. > - This means you cannot use the rfnoc_image_builder workflow to remove the > crossbar. You would need to generate your files with rfnoc_image_builder > -G, then manually remove the crossbar, then call make directly or do > rfnoc_image_builder --reuse. > - Like I said, not sure if this works at all. But it will definitely not > work if you don't observe these things: > - The primary connection to the device (from UHDs perspective) must have > access to an SEP with a ctrl interface. We also have a known issue where > the first SEP that UHD connects to needs to be the one with the ctrl > interface. > - I'm not sure about the route-not-found error. Like I said, you're doing > something way out of spec. > > --M > > On Wed, Jul 9, 2025 at 6:32 PM Brian Padalino <bpadalino@gmail.com> wrote: > >> I'm having a bit of a hard time understanding the minimal requirements >> for the CHDR Crossbar and connectivity. >> >> I'm working with an X440, so I have 3 transport adapters (int0, sfp0, >> sfp1), 2 blocks (radio0, radio1), 2 TX endpoints each with 4 ports (tx0, >> tx1), and 8 RX endpoints each with 1 port (rx[0-7]). >> >> I have tx0 ctrlport enabled, and none of the other ctrlports are enabled. >> I know I want rx[0-3] to only ever stream out of sfp0, and I want rx[4-7] >> to only ever stream out of sfp1. I want tx0 and tx1 to both receive CHDR >> packets from sfp0 and sfp1. I will always configure the device via int0. >> >> I also notice that rfnoc_core_kernel has a parameter for >> CHDR_XBAR_PRESENT. The comment for the parameter states: "1 if the CHDR >> crossbar is present. If 0 then transports are directly connected to SEPs". >> >> Connecting everything through the crossbar even with a sparse routing >> matrix ends up with around 18kLUT utilization. >> >> Since I know I want this extremely fixed and rigid design, I've got some >> questions: >> >> - How much of the CHDR crossbar can I remove? Can I get rid of it >> altogether? Are there any examples of a design with no CHDR crossbar? >> >> - Can I combine the RX SEPs into a single port per SFP connection using >> an AXI-Streaming mux of some type? Or is this accomplished in the same way >> in the crossbar with a sparse routing matrix? >> >> - How would one connect the multiple SEPs directly to the TA without >> going through the crossbar as the CHDR_XBAR_PRESENT parameter suggests is >> possible? Is it possible to describe this in the yaml file or does it >> require hand editing the generated rfnoc_image_core.sv file? >> >> - Since configuration is happening from int0, and tx0 is the only SEP >> with a ctrlport on it, does this suggest I need int0 to only be connected >> to tx0 in the connections and it doesn't need to go anywhere else? I will >> note that I tried this and I received a message saying a route couldn't be >> found for my remote streams. Is this maybe an oversight with remote >> streaming and sparse connectivity in the crossbar? >> >> I appreciate any insights you might be able to give. >> >> Thanks, >> Brian >> _______________________________________________ >> USRP-users mailing list -- usrp-users@lists.ettus.com >> To unsubscribe send an email to usrp-users-leave@lists.ettus.com >> > _______________________________________________ > USRP-users mailing list -- usrp-users@lists.ettus.com > To unsubscribe send an email to usrp-users-leave@lists.ettus.com >

Yeah, maybe. Can you post a bug on github with a minimal image core YAML that should work, but doesn't? Thanks, M On Thu, Jul 10, 2025 at 4:48 PM Brian Padalino <bpadalino@gmail.com> wrote: > Hey Martin, > > Thanks for the clarification. I understand I'm pushing a lot of boundaries > and I appreciate the time you're putting in to explain. > > Going back to the route-not-found error, this configuration is not > necessarily out of spec, I feel. I probably wasn't clear with the intention > there. For the current state of how I have things working: > > - The arm runs a python script to initialize the RFNoC graph and start > streams. This all goes through int0. > - SEP tx0 has a control port and is able to communicate the > configurations and is the first SEP in the list > - I have 8 RX SEP rx0-7, where rx0-3 have a connection in the crossbar > to sfp0 and rx4-7 have a connection in the crossbar to sfp1 > - All 8 RX SEP have a connection to and from int0 > - All SEPs are being set to remotely stream > > My issue arises when I remove the RX SEP connections to/from int0 - I get > that route-not-found error. My inclination is that since the RX SEPs only > have an output data port and no ctrl port, it should be fine to only > connect them in the crossbar to the TA they need to send their data. > > Do you think it's possible there is a bug in UHD when validating the > destination is reachable it is not considering a remote streaming > connection and is trying to connect back up to the SW endpoint? > > Thanks, > Brian > > On Thu, Jul 10, 2025 at 6:19 AM Martin Braun <martin.braun@ettus.com> > wrote: > >> - The only way to combine multiple SEPs into one is if you mux/demux in >> software as well as in a block after the SEP. The virtual channel >> feature was so supposed to address this, but as you know, we never >> implemented it. >> - I think if you removed the crossbar, UHD/RFNoC wouldn't balk at first, >> since it does a dynamic topology discovery, but I'm still not sure >> everything would work. We had originally thought we would support any >> number of crossbars (0, 1, 2, ...) but then eventually decided we'll use >> the routing matrix feature instead, and always assume a crossbar. >> - This means you cannot use the rfnoc_image_builder workflow to remove >> the crossbar. You would need to generate your files with >> rfnoc_image_builder -G, then manually remove the crossbar, then call make >> directly or do rfnoc_image_builder --reuse. >> - Like I said, not sure if this works at all. But it will definitely not >> work if you don't observe these things: >> - The primary connection to the device (from UHDs perspective) must >> have access to an SEP with a ctrl interface. We also have a known issue >> where the first SEP that UHD connects to needs to be the one with the ctrl >> interface. >> - I'm not sure about the route-not-found error. Like I said, you're doing >> something way out of spec. >> >> --M >> >> On Wed, Jul 9, 2025 at 6:32 PM Brian Padalino <bpadalino@gmail.com> >> wrote: >> >>> I'm having a bit of a hard time understanding the minimal requirements >>> for the CHDR Crossbar and connectivity. >>> >>> I'm working with an X440, so I have 3 transport adapters (int0, sfp0, >>> sfp1), 2 blocks (radio0, radio1), 2 TX endpoints each with 4 ports (tx0, >>> tx1), and 8 RX endpoints each with 1 port (rx[0-7]). >>> >>> I have tx0 ctrlport enabled, and none of the other ctrlports are >>> enabled. I know I want rx[0-3] to only ever stream out of sfp0, and I want >>> rx[4-7] to only ever stream out of sfp1. I want tx0 and tx1 to both receive >>> CHDR packets from sfp0 and sfp1. I will always configure the device via >>> int0. >>> >>> I also notice that rfnoc_core_kernel has a parameter for >>> CHDR_XBAR_PRESENT. The comment for the parameter states: "1 if the CHDR >>> crossbar is present. If 0 then transports are directly connected to SEPs". >>> >>> Connecting everything through the crossbar even with a sparse routing >>> matrix ends up with around 18kLUT utilization. >>> >>> Since I know I want this extremely fixed and rigid design, I've got some >>> questions: >>> >>> - How much of the CHDR crossbar can I remove? Can I get rid of it >>> altogether? Are there any examples of a design with no CHDR crossbar? >>> >>> - Can I combine the RX SEPs into a single port per SFP connection >>> using an AXI-Streaming mux of some type? Or is this accomplished in the >>> same way in the crossbar with a sparse routing matrix? >>> >>> - How would one connect the multiple SEPs directly to the TA without >>> going through the crossbar as the CHDR_XBAR_PRESENT parameter suggests is >>> possible? Is it possible to describe this in the yaml file or does it >>> require hand editing the generated rfnoc_image_core.sv file? >>> >>> - Since configuration is happening from int0, and tx0 is the only SEP >>> with a ctrlport on it, does this suggest I need int0 to only be connected >>> to tx0 in the connections and it doesn't need to go anywhere else? I will >>> note that I tried this and I received a message saying a route couldn't be >>> found for my remote streams. Is this maybe an oversight with remote >>> streaming and sparse connectivity in the crossbar? >>> >>> I appreciate any insights you might be able to give. >>> >>> Thanks, >>> Brian >>> _______________________________________________ >>> USRP-users mailing list -- usrp-users@lists.ettus.com >>> To unsubscribe send an email to usrp-users-leave@lists.ettus.com >>> >> _______________________________________________ >> USRP-users mailing list -- usrp-users@lists.ettus.com >> To unsubscribe send an email to usrp-users-leave@lists.ettus.com >> >