Hi all, I've got a predicament. Here's some *background* before I describe my problem: We purchased a Netgear xs728T 10GB switch in order to network 16 N200 devices, as well as a processing computer (which has a 10G ethernet card). Our 16 N200 devices are going to be used in a radar configuration where they are all simultaneously either transmitting or receiving. They are all receiving coincident 10MHz and 1PPS signals from 2 octoclocks, which are both fed from a third clock to ensure coincident clocks. They are connected to the switch via cat6 cables ~7feet in length (this item: https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136) *The problem:* During testing we've found that the example program tx_bursts, when run on one USRP at a time with a modest bandwidth, the "Waiting for async burst ack....failed" message appears about 75% of the time. If I try to use two or more USRPs at a time, then it appears every time. The line used for tx_bursts was typically something like this:./tx_bursts --args="addr0=192.168.10.104" --freq 10e6 --rate=1e6 --channels="0" for a one usrp test. For a multiple usrp test, I used lines like this:./tx_bursts --args="addr0=192.168.10.101,addr1=192.168.10.102,addr2=192.168.10.103,addr3=192.168.10.100,addr4=192.168.10.112,addr5=192.168.10.113,addr6=192.168.10.114" --freq 10e6 --rate=1e6 --channels="0,1,2,3,4,5,6" *Some evidence & information: *We also have a 5 port 1G DLink DGS-1005G. We've tested 4 N200s with tx_bursts on this switch (same cabling) and it works flawlessly. We also have tested a second 10G switch, the 8 port XS708E with 7 N200s with tx_bursts and it works flawlessly. I have wireshark dumps taken via a third machine that is connected to mirrored ports on the xs728T switch. I have attached them in case anyone can tell me if they see something wrong besides the missing acks. The wireshark filter to use is : ip.src == 192.168.10.104 and ip.dst == 192.168.10.104 or ip.src == 192.168.10.104 and ip.dst == 192.168.10.99 The xs728t switch information is: Boot version is 1.0.0.0, SW version is 6.5.1.26 (latest as of testing). I contacted Netgear support, and they sent us a brand new switch, which exhibited the same problem. I'm wondering if there's anyone out there who has had similar issues? Is there anything I can do to get more information or get around this problem? Can anyone see what the root cause might be in the wireshark captures? Any help or pointers are appreciated. Thank you, Kevin

 Hi all,

 I've got a predicament. Here's some background before I describe my problem:
 We purchased a Netgear xs728T 10GB switch in order to network 16 N200 devices, as well as a processing computer (which has a 10G ethernet card). Our 16 N200 devices are going to be used in a radar configuration where they are all simultaneously either transmitting or receiving. They are all receiving coincident 10MHz and 1PPS signals from 2 octoclocks, which are both fed from a third clock to ensure coincident clocks. They are connected to the switch via cat6 cables ~7feet in length (this item: https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136 https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136 )

 The problem:
 During testing we've found that the example program tx_bursts, when run on one USRP at a time with a modest bandwidth, the "Waiting for async burst ack....failed" message appears about 75% of the time. If I try to use two or more USRPs at a time, then it appears every time.
 The line used for tx_bursts was typically something like this: ./tx_bursts --args="addr0=192.168.10.104" --freq 10e6 --rate=1e6 --channels="0"
 for a one usrp test. For a multiple usrp test, I used lines like this: ./tx_bursts --args="addr0=192.168.10.101,addr1=192.168.10.102,addr2=192.168.10.103,addr3=192.168.10.100,addr4=192.168.10.112,addr5=192.168.10.113,addr6=192.168.10.114" --freq 10e6 --rate=1e6 --channels="0,1,2,3,4,5,6"


 Some evidence & information:
 We also have a 5 port 1G DLink DGS-1005G. We've tested 4 N200s with tx_bursts on this switch (same cabling) and it works flawlessly.
 We also have tested a second 10G switch, the 8 port XS708E with 7 N200s with tx_bursts and it works flawlessly.

 I have wireshark dumps taken via a third machine that is connected to mirrored ports on the xs728T switch. I have attached them in case anyone can tell me if they see something wrong besides the missing acks.
 The wireshark filter to use is : ip.src == 192.168.10.104 and ip.dst == 192.168.10.104 or ip.src == 192.168.10.104 and ip.dst == 192.168.10.99

 The xs728t switch information is: Boot version is 1.0.0.0, SW version is 6.5.1.26 (latest as of testing).
 I contacted Netgear support, and they sent us a brand new switch, which exhibited the same problem.

 I'm wondering if there's anyone out there who has had similar issues? Is there anything I can do to get more information or get around this problem?
 Can anyone see what the root cause might be in the wireshark captures?

 Any help or pointers are appreciated. Thank you,
 Kevin

 USRP-users mailing list
 USRP-users@lists.ettus.com
 http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

The only thing that jumps out to me is they are all on the same subnet... That is something I have typically avoided, but if you say it works in another configuration, that may not be the issue... > On May 24, 2017 at 12:12 PM Kevin Krieger via USRP-users <usrp-users@lists.ettus.com> wrote: > > Hi all, > > I've got a predicament. Here's some background before I describe my problem: > We purchased a Netgear xs728T 10GB switch in order to network 16 N200 devices, as well as a processing computer (which has a 10G ethernet card). Our 16 N200 devices are going to be used in a radar configuration where they are all simultaneously either transmitting or receiving. They are all receiving coincident 10MHz and 1PPS signals from 2 octoclocks, which are both fed from a third clock to ensure coincident clocks. They are connected to the switch via cat6 cables ~7feet in length (this item: https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136 https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136 ) > > The problem: > During testing we've found that the example program tx_bursts, when run on one USRP at a time with a modest bandwidth, the "Waiting for async burst ack....failed" message appears about 75% of the time. If I try to use two or more USRPs at a time, then it appears every time. > The line used for tx_bursts was typically something like this: ./tx_bursts --args="addr0=192.168.10.104" --freq 10e6 --rate=1e6 --channels="0" > for a one usrp test. For a multiple usrp test, I used lines like this: ./tx_bursts --args="addr0=192.168.10.101,addr1=192.168.10.102,addr2=192.168.10.103,addr3=192.168.10.100,addr4=192.168.10.112,addr5=192.168.10.113,addr6=192.168.10.114" --freq 10e6 --rate=1e6 --channels="0,1,2,3,4,5,6" > > > Some evidence & information: > We also have a 5 port 1G DLink DGS-1005G. We've tested 4 N200s with tx_bursts on this switch (same cabling) and it works flawlessly. > We also have tested a second 10G switch, the 8 port XS708E with 7 N200s with tx_bursts and it works flawlessly. > > I have wireshark dumps taken via a third machine that is connected to mirrored ports on the xs728T switch. I have attached them in case anyone can tell me if they see something wrong besides the missing acks. > The wireshark filter to use is : ip.src == 192.168.10.104 and ip.dst == 192.168.10.104 or ip.src == 192.168.10.104 and ip.dst == 192.168.10.99 > > The xs728t switch information is: Boot version is 1.0.0.0, SW version is 6.5.1.26 (latest as of testing). > I contacted Netgear support, and they sent us a brand new switch, which exhibited the same problem. > > I'm wondering if there's anyone out there who has had similar issues? Is there anything I can do to get more information or get around this problem? > Can anyone see what the root cause might be in the wireshark captures? > > Any help or pointers are appreciated. Thank you, > Kevin > > _______________________________________________ > USRP-users mailing list > USRP-users@lists.ettus.com > http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >

Hi Kevin, Hi Robin, the same-subnet configuration is the right one, here. There's no routing ambiguity if you have one card that serves the whole switch. Kevin, this might be much to ask, but: I'd ask you to both install wireshark and its development headers (typically, wireshark-dev or wireshark-devel) from your operating system's package manager. Then, get the UHD source code, and cd /path/to/uhd/tools/dissectors mkdir build cd build cmake -DETTUS_DISSECTOR_NAME=zpu .. make -j4 && make install Ideally, add your user to the group that has access to the raw network hardware (in case of my OS, that was the "wireshark" group), log out and back in. If that doesn't work, you have to record as root and analyze as your user (the USRP protocol dissectors got installed to your home directory). Then, record the traffic on your 10G interface, and analyze the packets in question as UHD (some/many will be sample packets, i.e. CVITA). Best regards, Marcus On 24.05.2017 20:15, ROBIN TORTORA via USRP-users wrote: > > The only thing that jumps out to me is they are all on the same subnet... > > > That is something I have typically avoided, but if you say it works in > another configuration, that may not be the issue... > >> On May 24, 2017 at 12:12 PM Kevin Krieger via USRP-users >> <usrp-users@lists.ettus.com> wrote: >> >> Hi all, >> >> I've got a predicament. Here's some *background* before I describe my >> problem: >> We purchased a Netgear xs728T 10GB switch in order to network 16 N200 >> devices, as well as a processing computer (which has a 10G ethernet >> card). Our 16 N200 devices are going to be used in a radar >> configuration where they are all simultaneously either transmitting >> or receiving. They are all receiving coincident 10MHz and 1PPS >> signals from 2 octoclocks, which are both fed from a third clock to >> ensure coincident clocks. They are connected to the switch via cat6 >> cables ~7feet in length (this item: >> https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136) >> >> *The problem:* >> During testing we've found that the example program tx_bursts, when >> run on one USRP at a time with a modest bandwidth, the "Waiting for >> async burst ack....failed" message appears about 75% of the time. If >> I try to use two or more USRPs at a time, then it appears every time. >> The line used for tx_bursts was typically something like >> this:./tx_bursts --args="addr0=192.168.10.104" --freq 10e6 --rate=1e6 >> --channels="0" >> for a one usrp test. For a multiple usrp test, I used lines like >> this:./tx_bursts >> --args="addr0=192.168.10.101,addr1=192.168.10.102,addr2=192.168.10.103,addr3=192.168.10.100,addr4=192.168.10.112,addr5=192.168.10.113,addr6=192.168.10.114" >> --freq 10e6 --rate=1e6 --channels="0,1,2,3,4,5,6" >> >> >> *Some evidence & information: >> *We also have a 5 port 1G DLink DGS-1005G. We've tested 4 N200s with >> tx_bursts on this switch (same cabling) and it works flawlessly. >> We also have tested a second 10G switch, the 8 port XS708E with 7 >> N200s with tx_bursts and it works flawlessly. >> >> I have wireshark dumps taken via a third machine that is connected to >> mirrored ports on the xs728T switch. I have attached them in case >> anyone can tell me if they see something wrong besides the missing acks. >> The wireshark filter to use is : ip.src == 192.168.10.104 and ip.dst >> == 192.168.10.104 or ip.src == 192.168.10.104 and ip.dst == 192.168.10.99 >> >> The xs728t switch information is: Boot version is 1.0.0.0, SW version >> is 6.5.1.26 (latest as of testing). >> I contacted Netgear support, and they sent us a brand new switch, >> which exhibited the same problem. >> >> I'm wondering if there's anyone out there who has had similar issues? >> Is there anything I can do to get more information or get around this >> problem? >> Can anyone see what the root cause might be in the wireshark captures? >> >> Any help or pointers are appreciated. Thank you, >> Kevin > > > > >> _______________________________________________ >> USRP-users mailing list >> USRP-users@lists.ettus.com >> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com > > > _______________________________________________ > USRP-users mailing list > USRP-users@lists.ettus.com > http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

Hi Marcus, Thank you - I was sure that the same subnet was correct but it's good to have confirmation. I have done the dissecting previously, and I recall getting a bunch of "huh what's that?"' messages and then a 'wassup bro' or something but I'll do it again and let you know what I see. (Hilarious descriptions btw!) Kevin On 05/24/2017 08:07 PM, Marcus Müller via USRP-users wrote: > > Hi Kevin, Hi Robin, > > the same-subnet configuration is the right one, here. There's no > routing ambiguity if you have one card that serves the whole switch. > > Kevin, this might be much to ask, but: I'd ask you to both install > wireshark and its development headers (typically, wireshark-dev or > wireshark-devel) from your operating system's package manager. > > Then, get the UHD source code, and > > cd /path/to/uhd/tools/dissectors > mkdir build > cd build > cmake -DETTUS_DISSECTOR_NAME=zpu .. > make -j4 && make install > > Ideally, add your user to the group that has access to the raw network > hardware (in case of my OS, that was the "wireshark" group), log out > and back in. > > If that doesn't work, you have to record as root and analyze as your > user (the USRP protocol dissectors got installed to your home directory). > > Then, record the traffic on your 10G interface, and analyze the > packets in question as UHD (some/many will be sample packets, i.e. CVITA). > > Best regards, > > Marcus > > On 24.05.2017 20:15, ROBIN TORTORA via USRP-users wrote: >> >> The only thing that jumps out to me is they are all on the same subnet... >> >> >> That is something I have typically avoided, but if you say it works >> in another configuration, that may not be the issue... >> >>> On May 24, 2017 at 12:12 PM Kevin Krieger via USRP-users >>> <usrp-users@lists.ettus.com> wrote: >>> >>> Hi all, >>> >>> I've got a predicament. Here's some *background* before I describe >>> my problem: >>> We purchased a Netgear xs728T 10GB switch in order to network 16 >>> N200 devices, as well as a processing computer (which has a 10G >>> ethernet card). Our 16 N200 devices are going to be used in a radar >>> configuration where they are all simultaneously either transmitting >>> or receiving. They are all receiving coincident 10MHz and 1PPS >>> signals from 2 octoclocks, which are both fed from a third clock to >>> ensure coincident clocks. They are connected to the switch via cat6 >>> cables ~7feet in length (this item: >>> https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136) >>> >>> *The problem:* >>> During testing we've found that the example program tx_bursts, when >>> run on one USRP at a time with a modest bandwidth, the "Waiting for >>> async burst ack....failed" message appears about 75% of the time. If >>> I try to use two or more USRPs at a time, then it appears every time. >>> The line used for tx_bursts was typically something like >>> this:./tx_bursts --args="addr0=192.168.10.104" --freq 10e6 >>> --rate=1e6 --channels="0" >>> for a one usrp test. For a multiple usrp test, I used lines like >>> this:./tx_bursts >>> --args="addr0=192.168.10.101,addr1=192.168.10.102,addr2=192.168.10.103,addr3=192.168.10.100,addr4=192.168.10.112,addr5=192.168.10.113,addr6=192.168.10.114" >>> --freq 10e6 --rate=1e6 --channels="0,1,2,3,4,5,6" >>> >>> >>> *Some evidence & information: >>> *We also have a 5 port 1G DLink DGS-1005G. We've tested 4 N200s with >>> tx_bursts on this switch (same cabling) and it works flawlessly. >>> We also have tested a second 10G switch, the 8 port XS708E with 7 >>> N200s with tx_bursts and it works flawlessly. >>> >>> I have wireshark dumps taken via a third machine that is connected >>> to mirrored ports on the xs728T switch. I have attached them in case >>> anyone can tell me if they see something wrong besides the missing acks. >>> The wireshark filter to use is : ip.src == 192.168.10.104 and ip.dst >>> == 192.168.10.104 or ip.src == 192.168.10.104 and ip.dst == >>> 192.168.10.99 >>> >>> The xs728t switch information is: Boot version is 1.0.0.0, SW >>> version is 6.5.1.26 (latest as of testing). >>> I contacted Netgear support, and they sent us a brand new switch, >>> which exhibited the same problem. >>> >>> I'm wondering if there's anyone out there who has had similar >>> issues? Is there anything I can do to get more information or get >>> around this problem? >>> Can anyone see what the root cause might be in the wireshark captures? >>> >>> Any help or pointers are appreciated. Thank you, >>> Kevin >> >> >>> _______________________________________________ >>> USRP-users mailing list >>> USRP-users@lists.ettus.com >>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >> >> >> _______________________________________________ >> USRP-users mailing list >> USRP-users@lists.ettus.com >> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com > > > > _______________________________________________ > USRP-users mailing list > USRP-users@lists.ettus.com > http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

Hi Marcus, I finally got around to this. I've attached 9 captures. The setup for the captures was: XS728T with latest firmware running, factory default Computer running tx_bursts has IP of 192.168.10.60 or .67 N200 has IP of 192.168.10.107 ./tx_bursts --args addr0=192.168.10.107 --freq 10e6 --rate 1e6 Captures 0,1 and 2 are with both the ports (n200 side, and computer side) mirrored to the wireshark capture port, so you essentially see duplicate traffic. Captures 3,4 and 5 are with only the computer side traffic captured Captures 6,7 and 8 are with only the N200 side traffic captured. Captures 0, 4, and 8 are with the messages "Sent packet: 363 samples" appearing, but then the "Waiting for async burst ACK ... fail" message also appears. Captures 1, 3 and 7 are with the message :Error: Runtimeerror: fifo ctrl timed out looking for acks" appearing. Captures 2, 5 and 6 are with successful runs of the tx bursts program, so instead of 'fail' at the end there is 'success'. There are txt files with the output of the program for each capture, as well as csv files showing the dissected output (only found uhd in the dissectors). Can you see anything wrong with these runs? Any help is appreciated. Thanks, Kevin On 06/15/2017 02:28 PM, Kevin Krieger via USRP-users wrote: > Hi Marcus, > > Thank you - I was sure that the same subnet was correct but it's good > to have confirmation. > I have done the dissecting previously, and I recall getting a bunch of > "huh what's that?"' messages and then a 'wassup bro' or something but > I'll do it again and let you know what I see. (Hilarious descriptions > btw!) > > Kevin > > > On 05/24/2017 08:07 PM, Marcus Müller via USRP-users wrote: >> >> Hi Kevin, Hi Robin, >> >> the same-subnet configuration is the right one, here. There's no >> routing ambiguity if you have one card that serves the whole switch. >> >> Kevin, this might be much to ask, but: I'd ask you to both install >> wireshark and its development headers (typically, wireshark-dev or >> wireshark-devel) from your operating system's package manager. >> >> Then, get the UHD source code, and >> >> cd /path/to/uhd/tools/dissectors >> mkdir build >> cd build >> cmake -DETTUS_DISSECTOR_NAME=zpu .. >> make -j4 && make install >> >> Ideally, add your user to the group that has access to the raw >> network hardware (in case of my OS, that was the "wireshark" group), >> log out and back in. >> >> If that doesn't work, you have to record as root and analyze as your >> user (the USRP protocol dissectors got installed to your home directory). >> >> Then, record the traffic on your 10G interface, and analyze the >> packets in question as UHD (some/many will be sample packets, i.e. >> CVITA). >> >> Best regards, >> >> Marcus >> >> On 24.05.2017 20:15, ROBIN TORTORA via USRP-users wrote: >>> >>> The only thing that jumps out to me is they are all on the same >>> subnet... >>> >>> >>> That is something I have typically avoided, but if you say it works >>> in another configuration, that may not be the issue... >>> >>>> On May 24, 2017 at 12:12 PM Kevin Krieger via USRP-users >>>> <usrp-users@lists.ettus.com> wrote: >>>> >>>> Hi all, >>>> >>>> I've got a predicament. Here's some *background* before I describe >>>> my problem: >>>> We purchased a Netgear xs728T 10GB switch in order to network 16 >>>> N200 devices, as well as a processing computer (which has a 10G >>>> ethernet card). Our 16 N200 devices are going to be used in a radar >>>> configuration where they are all simultaneously either transmitting >>>> or receiving. They are all receiving coincident 10MHz and 1PPS >>>> signals from 2 octoclocks, which are both fed from a third clock to >>>> ensure coincident clocks. They are connected to the switch via cat6 >>>> cables ~7feet in length (this item: >>>> https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136) >>>> >>>> *The problem:* >>>> During testing we've found that the example program tx_bursts, when >>>> run on one USRP at a time with a modest bandwidth, the "Waiting for >>>> async burst ack....failed" message appears about 75% of the time. >>>> If I try to use two or more USRPs at a time, then it appears every >>>> time. >>>> The line used for tx_bursts was typically something like >>>> this:./tx_bursts --args="addr0=192.168.10.104" --freq 10e6 >>>> --rate=1e6 --channels="0" >>>> for a one usrp test. For a multiple usrp test, I used lines like >>>> this:./tx_bursts >>>> --args="addr0=192.168.10.101,addr1=192.168.10.102,addr2=192.168.10.103,addr3=192.168.10.100,addr4=192.168.10.112,addr5=192.168.10.113,addr6=192.168.10.114" >>>> --freq 10e6 --rate=1e6 --channels="0,1,2,3,4,5,6" >>>> >>>> >>>> *Some evidence & information: >>>> *We also have a 5 port 1G DLink DGS-1005G. We've tested 4 N200s >>>> with tx_bursts on this switch (same cabling) and it works flawlessly. >>>> We also have tested a second 10G switch, the 8 port XS708E with 7 >>>> N200s with tx_bursts and it works flawlessly. >>>> >>>> I have wireshark dumps taken via a third machine that is connected >>>> to mirrored ports on the xs728T switch. I have attached them in >>>> case anyone can tell me if they see something wrong besides the >>>> missing acks. >>>> The wireshark filter to use is : ip.src == 192.168.10.104 and >>>> ip.dst == 192.168.10.104 or ip.src == 192.168.10.104 and ip.dst == >>>> 192.168.10.99 >>>> >>>> The xs728t switch information is: Boot version is 1.0.0.0, SW >>>> version is 6.5.1.26 (latest as of testing). >>>> I contacted Netgear support, and they sent us a brand new switch, >>>> which exhibited the same problem. >>>> >>>> I'm wondering if there's anyone out there who has had similar >>>> issues? Is there anything I can do to get more information or get >>>> around this problem? >>>> Can anyone see what the root cause might be in the wireshark captures? >>>> >>>> Any help or pointers are appreciated. Thank you, >>>> Kevin >>> >>> >>>> _______________________________________________ >>>> USRP-users mailing list >>>> USRP-users@lists.ettus.com >>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>> >>> >>> _______________________________________________ >>> USRP-users mailing list >>> USRP-users@lists.ettus.com >>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >> >> >> >> _______________________________________________ >> USRP-users mailing list >> USRP-users@lists.ettus.com >> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com > > > > _______________________________________________ > USRP-users mailing list > USRP-users@lists.ettus.com > http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

Hi, Any chance to look at these wireshark captures? Thanks, Kevin On 10/30/2017 05:30 PM, Kevin Krieger via USRP-users wrote: > Hi Marcus, > > I finally got around to this. > I've attached 9 captures. > > The setup for the captures was: > XS728T with latest firmware running, factory default > Computer running tx_bursts has IP of 192.168.10.60 or .67 > N200 has IP of 192.168.10.107 > ./tx_bursts --args addr0=192.168.10.107 --freq 10e6 --rate 1e6 > > Captures 0,1 and 2 are with both the ports (n200 side, and computer > side) mirrored to the wireshark capture port, so you essentially see > duplicate traffic. > Captures 3,4 and 5 are with only the computer side traffic captured > Captures 6,7 and 8 are with only the N200 side traffic captured. > > Captures 0, 4, and 8 are with the messages "Sent packet: 363 samples" > appearing, but then the "Waiting for async burst ACK ... fail" message > also appears. > Captures 1, 3 and 7 are with the message :Error: Runtimeerror: fifo > ctrl timed out looking for acks" appearing. > Captures 2, 5 and 6 are with successful runs of the tx bursts program, > so instead of 'fail' at the end there is 'success'. > > There are txt files with the output of the program for each capture, > as well as csv files showing the dissected output (only found uhd in > the dissectors). > > Can you see anything wrong with these runs? Any help is appreciated. > > Thanks, > Kevin > > > On 06/15/2017 02:28 PM, Kevin Krieger via USRP-users wrote: >> Hi Marcus, >> >> Thank you - I was sure that the same subnet was correct but it's good >> to have confirmation. >> I have done the dissecting previously, and I recall getting a bunch >> of "huh what's that?"' messages and then a 'wassup bro' or something >> but I'll do it again and let you know what I see. (Hilarious >> descriptions btw!) >> >> Kevin >> >> >> On 05/24/2017 08:07 PM, Marcus Müller via USRP-users wrote: >>> >>> Hi Kevin, Hi Robin, >>> >>> the same-subnet configuration is the right one, here. There's no >>> routing ambiguity if you have one card that serves the whole switch. >>> >>> Kevin, this might be much to ask, but: I'd ask you to both install >>> wireshark and its development headers (typically, wireshark-dev or >>> wireshark-devel) from your operating system's package manager. >>> >>> Then, get the UHD source code, and >>> >>> cd /path/to/uhd/tools/dissectors >>> mkdir build >>> cd build >>> cmake -DETTUS_DISSECTOR_NAME=zpu .. >>> make -j4 && make install >>> >>> Ideally, add your user to the group that has access to the raw >>> network hardware (in case of my OS, that was the "wireshark" group), >>> log out and back in. >>> >>> If that doesn't work, you have to record as root and analyze as your >>> user (the USRP protocol dissectors got installed to your home >>> directory). >>> >>> Then, record the traffic on your 10G interface, and analyze the >>> packets in question as UHD (some/many will be sample packets, i.e. >>> CVITA). >>> >>> Best regards, >>> >>> Marcus >>> >>> On 24.05.2017 20:15, ROBIN TORTORA via USRP-users wrote: >>>> >>>> The only thing that jumps out to me is they are all on the same >>>> subnet... >>>> >>>> >>>> That is something I have typically avoided, but if you say it works >>>> in another configuration, that may not be the issue... >>>> >>>>> On May 24, 2017 at 12:12 PM Kevin Krieger via USRP-users >>>>> <usrp-users@lists.ettus.com> wrote: >>>>> >>>>> Hi all, >>>>> >>>>> I've got a predicament. Here's some *background* before I describe >>>>> my problem: >>>>> We purchased a Netgear xs728T 10GB switch in order to network 16 >>>>> N200 devices, as well as a processing computer (which has a 10G >>>>> ethernet card). Our 16 N200 devices are going to be used in a >>>>> radar configuration where they are all simultaneously either >>>>> transmitting or receiving. They are all receiving coincident 10MHz >>>>> and 1PPS signals from 2 octoclocks, which are both fed from a >>>>> third clock to ensure coincident clocks. They are connected to the >>>>> switch via cat6 cables ~7feet in length (this item: >>>>> https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136) >>>>> >>>>> *The problem:* >>>>> During testing we've found that the example program tx_bursts, >>>>> when run on one USRP at a time with a modest bandwidth, the >>>>> "Waiting for async burst ack....failed" message appears about 75% >>>>> of the time. If I try to use two or more USRPs at a time, then it >>>>> appears every time. >>>>> The line used for tx_bursts was typically something like >>>>> this:./tx_bursts --args="addr0=192.168.10.104" --freq 10e6 >>>>> --rate=1e6 --channels="0" >>>>> for a one usrp test. For a multiple usrp test, I used lines like >>>>> this:./tx_bursts >>>>> --args="addr0=192.168.10.101,addr1=192.168.10.102,addr2=192.168.10.103,addr3=192.168.10.100,addr4=192.168.10.112,addr5=192.168.10.113,addr6=192.168.10.114" >>>>> --freq 10e6 --rate=1e6 --channels="0,1,2,3,4,5,6" >>>>> >>>>> >>>>> *Some evidence & information: >>>>> *We also have a 5 port 1G DLink DGS-1005G. We've tested 4 N200s >>>>> with tx_bursts on this switch (same cabling) and it works flawlessly. >>>>> We also have tested a second 10G switch, the 8 port XS708E with 7 >>>>> N200s with tx_bursts and it works flawlessly. >>>>> >>>>> I have wireshark dumps taken via a third machine that is connected >>>>> to mirrored ports on the xs728T switch. I have attached them in >>>>> case anyone can tell me if they see something wrong besides the >>>>> missing acks. >>>>> The wireshark filter to use is : ip.src == 192.168.10.104 and >>>>> ip.dst == 192.168.10.104 or ip.src == 192.168.10.104 and ip.dst == >>>>> 192.168.10.99 >>>>> >>>>> The xs728t switch information is: Boot version is 1.0.0.0, SW >>>>> version is 6.5.1.26 (latest as of testing). >>>>> I contacted Netgear support, and they sent us a brand new switch, >>>>> which exhibited the same problem. >>>>> >>>>> I'm wondering if there's anyone out there who has had similar >>>>> issues? Is there anything I can do to get more information or get >>>>> around this problem? >>>>> Can anyone see what the root cause might be in the wireshark captures? >>>>> >>>>> Any help or pointers are appreciated. Thank you, >>>>> Kevin >>>> >>>> >>>>> _______________________________________________ >>>>> USRP-users mailing list >>>>> USRP-users@lists.ettus.com >>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>>> >>>> >>>> _______________________________________________ >>>> USRP-users mailing list >>>> USRP-users@lists.ettus.com >>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>> >>> >>> >>> _______________________________________________ >>> USRP-users mailing list >>> USRP-users@lists.ettus.com >>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >> >> >> >> _______________________________________________ >> USRP-users mailing list >> USRP-users@lists.ettus.com >> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com > > > > _______________________________________________ > USRP-users mailing list > USRP-users@lists.ettus.com > http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

Kevin, I glanced through Captures 0,1,2 and could see the general gist of where it all goes wrong, but it’s not really clear why. The dissector seems out of date, at least it isn’t decoding correctly but I can still follow it. Capture 0: Packets 2551 through 2606 are the actual sample data packets. Then there is an approx 1.2second pause before UHD, having timed out waiting for ACK’s begins some basic control interaction initiated by the host. Contrast that with Capture 2, where you will see a nicely paced stream of ACK’s (starts at packet 2609) flowing back from the USRP at exact intervals pacing the real time TX operation as the DSP consumes the samples. Capture 1: This goes to the weeds early, streaming never gets close to starting. At around packet 1324 you suddenly see a series of pauses for exactly 0.5 secs each. So a couple of thoughts bordering on guesses: Rate conversion in switches is always a little bit of a minefield when you have a single stream that is faster than the skinny pipe. In this case, operations like the sample burst are streaming from the host faster than 1Gbs into the switch forcing the switch to buffer samples. It’s surprising sometimes how quick these lower cost single chip switches can get buffer starved. Try using 'ethtool' on the Linux host to force the host port to 1Gbps to eliminate rate conversion in the switch as a cause. N210 in general is the most reliable of all USRP’s but it has one quirk, which is that the ethernet port is forced to be 1Gbps and it does no auto negotiation. This often trips up people who connect it to a 100mbit switch or host. I have never actually tried connecting it to 10GBase-T switches…there may be some (switch implementation specific) gotcha’s. Mirroring multiple ports to a single port can affect how the problem manifests because you create new bottle necks in the switch …if the 10G switch is suspect then I suggest interposing a cheap 1G switch that supports mirroring between USRP and 10G switch, that way you’ll have higher confidence exactly what and when went onto the wire. This is also an interesting experiment because it isolates the N210’s ethernet port from the 10G switch…curious to see if it now starts to work OK. I have had this happen when debugging other products. The ICMP port unreachable is likely a red herring, some spurious Linux service rather than UHD originating those packets I suspect. Might be worth looking at the switch internal counters to see if any physical layer error counters are increasing. No smoking gun I’m afraid; your fall back plan might be to stuff quad port 1G cards in your host, they’re cheap if you have the PCIe slots free. -Ian > On Nov 6, 2017, at 3:13 PM, Kevin Krieger via USRP-users <usrp-users@lists.ettus.com> wrote: > > Hi, > > Any chance to look at these wireshark captures? > > Thanks, > Kevin > > On 10/30/2017 05:30 PM, Kevin Krieger via USRP-users wrote: >> Hi Marcus, >> >> I finally got around to this. >> I've attached 9 captures. >> >> The setup for the captures was: >> XS728T with latest firmware running, factory default >> Computer running tx_bursts has IP of 192.168.10.60 or .67 >> N200 has IP of 192.168.10.107 >> ./tx_bursts --args addr0=192.168.10.107 --freq 10e6 --rate 1e6 >> >> Captures 0,1 and 2 are with both the ports (n200 side, and computer side) mirrored to the wireshark capture port, so you essentially see duplicate traffic. >> Captures 3,4 and 5 are with only the computer side traffic captured >> Captures 6,7 and 8 are with only the N200 side traffic captured. >> >> Captures 0, 4, and 8 are with the messages "Sent packet: 363 samples" appearing, but then the "Waiting for async burst ACK ... fail" message also appears. >> Captures 1, 3 and 7 are with the message :Error: Runtimeerror: fifo ctrl timed out looking for acks" appearing. >> Captures 2, 5 and 6 are with successful runs of the tx bursts program, so instead of 'fail' at the end there is 'success'. >> >> There are txt files with the output of the program for each capture, as well as csv files showing the dissected output (only found uhd in the dissectors). >> >> Can you see anything wrong with these runs? Any help is appreciated. >> >> Thanks, >> Kevin >> >> >> On 06/15/2017 02:28 PM, Kevin Krieger via USRP-users wrote: >>> Hi Marcus, >>> >>> Thank you - I was sure that the same subnet was correct but it's good to have confirmation. >>> I have done the dissecting previously, and I recall getting a bunch of "huh what's that?"' messages and then a 'wassup bro' or something but I'll do it again and let you know what I see. (Hilarious descriptions btw!) >>> >>> Kevin >>> >>> >>> On 05/24/2017 08:07 PM, Marcus Müller via USRP-users wrote: >>>> Hi Kevin, Hi Robin, >>>> the same-subnet configuration is the right one, here. There's no routing ambiguity if you have one card that serves the whole switch. >>>> Kevin, this might be much to ask, but: I'd ask you to both install wireshark and its development headers (typically, wireshark-dev or wireshark-devel) from your operating system's package manager. >>>> Then, get the UHD source code, and >>>> >>>> cd /path/to/uhd/tools/dissectors >>>> mkdir build >>>> cd build >>>> cmake -DETTUS_DISSECTOR_NAME=zpu .. >>>> make -j4 && make install >>>> >>>> Ideally, add your user to the group that has access to the raw network hardware (in case of my OS, that was the "wireshark" group), log out and back in. >>>> >>>> If that doesn't work, you have to record as root and analyze as your user (the USRP protocol dissectors got installed to your home directory). >>>> >>>> Then, record the traffic on your 10G interface, and analyze the packets in question as UHD (some/many will be sample packets, i.e. CVITA). >>>> >>>> Best regards, >>>> >>>> Marcus >>>> On 24.05.2017 20:15, ROBIN TORTORA via USRP-users wrote: >>>>> The only thing that jumps out to me is they are all on the same subnet... >>>>> >>>>> >>>>> That is something I have typically avoided, but if you say it works in another configuration, that may not be the issue... >>>>> >>>>>> On May 24, 2017 at 12:12 PM Kevin Krieger via USRP-users <usrp-users@lists.ettus.com> <mailto:usrp-users@lists.ettus.com> wrote: >>>>>> >>>>>> Hi all, >>>>>> >>>>>> I've got a predicament. Here's some background before I describe my problem: >>>>>> We purchased a Netgear xs728T 10GB switch in order to network 16 N200 devices, as well as a processing computer (which has a 10G ethernet card). Our 16 N200 devices are going to be used in a radar configuration where they are all simultaneously either transmitting or receiving. They are all receiving coincident 10MHz and 1PPS signals from 2 octoclocks, which are both fed from a third clock to ensure coincident clocks. They are connected to the switch via cat6 cables ~7feet in length (this item: https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136 <https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136>) >>>>>> >>>>>> The problem: >>>>>> During testing we've found that the example program tx_bursts, when run on one USRP at a time with a modest bandwidth, the "Waiting for async burst ack....failed" message appears about 75% of the time. If I try to use two or more USRPs at a time, then it appears every time. >>>>>> The line used for tx_bursts was typically something like this: ./tx_bursts --args="addr0=192.168.10.104" --freq 10e6 --rate=1e6 --channels="0" >>>>>> for a one usrp test. For a multiple usrp test, I used lines like this: ./tx_bursts --args="addr0=192.168.10.101,addr1=192.168.10.102,addr2=192.168.10.103,addr3=192.168.10.100,addr4=192.168.10.112,addr5=192.168.10.113,addr6=192.168.10.114" --freq 10e6 --rate=1e6 --channels="0,1,2,3,4,5,6" >>>>>> >>>>>> >>>>>> Some evidence & information: >>>>>> We also have a 5 port 1G DLink DGS-1005G. We've tested 4 N200s with tx_bursts on this switch (same cabling) and it works flawlessly. >>>>>> We also have tested a second 10G switch, the 8 port XS708E with 7 N200s with tx_bursts and it works flawlessly. >>>>>> >>>>>> I have wireshark dumps taken via a third machine that is connected to mirrored ports on the xs728T switch. I have attached them in case anyone can tell me if they see something wrong besides the missing acks. >>>>>> The wireshark filter to use is : ip.src == 192.168.10.104 and ip.dst == 192.168.10.104 or ip.src == 192.168.10.104 and ip.dst == 192.168.10.99 >>>>>> >>>>>> The xs728t switch information is: Boot version is 1.0.0.0, SW version is 6.5.1.26 (latest as of testing). >>>>>> I contacted Netgear support, and they sent us a brand new switch, which exhibited the same problem. >>>>>> >>>>>> I'm wondering if there's anyone out there who has had similar issues? Is there anything I can do to get more information or get around this problem? >>>>>> Can anyone see what the root cause might be in the wireshark captures? >>>>>> >>>>>> Any help or pointers are appreciated. Thank you, >>>>>> Kevin >>>>> >>>>> >>>>> >>>>>> _______________________________________________ >>>>>> USRP-users mailing list >>>>>> USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> >>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com> >>>>> >>>>> >>>>> _______________________________________________ >>>>> USRP-users mailing list >>>>> USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> >>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com> >>>> >>>> >>>> >>>> _______________________________________________ >>>> USRP-users mailing list >>>> USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> >>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com> >>> >>> >>> >>> _______________________________________________ >>> USRP-users mailing list >>> USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> >>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com> >> >> >> >> _______________________________________________ >> USRP-users mailing list >> USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> >> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com> > > _______________________________________________ > USRP-users mailing list > USRP-users@lists.ettus.com > http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

Hi Ian, OK - is there code for the dissector available to build a new version? Or do you think it's worth it? Also - our host sending the samples was actually only 1Gbps ethernet (we weren't using the 10Gbps card for that), so I'm not sure that the samples would be going into the switch faster than this. We were using the 10G card for wireshark. Thanks for the info about the N210 ethernet port, maybe we can try forcing the switch ports to 1G and see if that helps. Also adding a 1G switch in between the USRP and 10G will be a good test if we can get our hands on a 1G switch. We discussed just getting four 4port 1G cards, we would need to upgrade our motherboard though. Thanks, If we figure anything out we'll post back here. Regards, Kevin On 11/09/2017 09:14 AM, Ian Buckley wrote: > > Kevin, > I glanced through Captures 0,1,2 and could see the general gist of > where it all goes wrong, but it’s not really clear why. The dissector > seems out of date, at least it isn’t decoding correctly but I can > still follow it. > > Capture 0: > Packets 2551 through 2606 are the actual sample data packets. > Then there is an approx 1.2second pause before UHD, having timed out > waiting for ACK’s begins some basic control interaction initiated by > the host. > > Contrast that with Capture 2, where you will see a nicely paced stream > of ACK’s (starts at packet 2609) flowing back from the USRP at exact > intervals pacing the real time TX operation as the DSP consumes the > samples. > > Capture 1: > This goes to the weeds early, streaming never gets close to starting. > At around packet 1324 you suddenly see a series of pauses for exactly > 0.5 secs each. > > So a couple of thoughts bordering on guesses: > Rate conversion in switches is always a little bit of a minefield when > you have a single stream that is faster than the skinny pipe. In this > case, operations like the sample burst are streaming from the host > faster than 1Gbs into the switch forcing the switch to buffer samples. > It’s surprising sometimes how quick these lower cost single chip > switches can get buffer starved. Try using 'ethtool' on the Linux host > to force the host port to 1Gbps to eliminate rate conversion in the > switch as a cause. > > N210 in general is the most reliable of all USRP’s but it has one > quirk, which is that the ethernet port is forced to be 1Gbps and it > does no auto negotiation. This often trips up people who connect it to > a 100mbit switch or host. I have never actually tried connecting it to > 10GBase-T switches…there may be some (switch implementation specific) > gotcha’s. > > Mirroring multiple ports to a single port can affect how the problem > manifests because you create new bottle necks in the switch …if the > 10G switch is suspect then I suggest interposing a cheap 1G switch > that supports mirroring between USRP and 10G switch, that way you’ll > have higher confidence exactly what and when went onto the wire. This > is also an interesting experiment because it isolates the N210’s > ethernet port from the 10G switch…curious to see if it now starts to > work OK. I have had this happen when debugging other products. > > The ICMP port unreachable is likely a red herring, some spurious Linux > service rather than UHD originating those packets I suspect. > > Might be worth looking at the switch internal counters to see if any > physical layer error counters are increasing. > > No smoking gun I’m afraid; your fall back plan might be to stuff quad > port 1G cards in your host, they’re cheap if you have the PCIe slots free. > > -Ian > >> On Nov 6, 2017, at 3:13 PM, Kevin Krieger via USRP-users >> <usrp-users@lists.ettus.com <mailto:usrp-users@lists.ettus.com>> wrote: >> >> Hi, >> >> Any chance to look at these wireshark captures? >> >> Thanks, >> Kevin >> >> On 10/30/2017 05:30 PM, Kevin Krieger via USRP-users wrote: >>> Hi Marcus, >>> >>> I finally got around to this. >>> I've attached 9 captures. >>> >>> The setup for the captures was: >>> XS728T with latest firmware running, factory default >>> Computer running tx_bursts has IP of 192.168.10.60 or .67 >>> N200 has IP of 192.168.10.107 >>> ./tx_bursts --args addr0=192.168.10.107 --freq 10e6 --rate 1e6 >>> >>> Captures 0,1 and 2 are with both the ports (n200 side, and computer >>> side) mirrored to the wireshark capture port, so you essentially see >>> duplicate traffic. >>> Captures 3,4 and 5 are with only the computer side traffic captured >>> Captures 6,7 and 8 are with only the N200 side traffic captured. >>> >>> Captures 0, 4, and 8 are with the messages "Sent packet: 363 >>> samples" appearing, but then the "Waiting for async burst ACK ... >>> fail" message also appears. >>> Captures 1, 3 and 7 are with the message :Error: Runtimeerror: fifo >>> ctrl timed out looking for acks" appearing. >>> Captures 2, 5 and 6 are with successful runs of the tx bursts >>> program, so instead of 'fail' at the end there is 'success'. >>> >>> There are txt files with the output of the program for each capture, >>> as well as csv files showing the dissected output (only found uhd in >>> the dissectors). >>> >>> Can you see anything wrong with these runs? Any help is appreciated. >>> >>> Thanks, >>> Kevin >>> >>> >>> On 06/15/2017 02:28 PM, Kevin Krieger via USRP-users wrote: >>>> Hi Marcus, >>>> >>>> Thank you - I was sure that the same subnet was correct but it's >>>> good to have confirmation. >>>> I have done the dissecting previously, and I recall getting a bunch >>>> of "huh what's that?"' messages and then a 'wassup bro' or >>>> something but I'll do it again and let you know what I see. >>>> (Hilarious descriptions btw!) >>>> >>>> Kevin >>>> >>>> >>>> On 05/24/2017 08:07 PM, Marcus Müller via USRP-users wrote: >>>>> >>>>> Hi Kevin, Hi Robin, >>>>> >>>>> the same-subnet configuration is the right one, here. There's no >>>>> routing ambiguity if you have one card that serves the whole switch. >>>>> >>>>> Kevin, this might be much to ask, but: I'd ask you to both install >>>>> wireshark and its development headers (typically, wireshark-dev or >>>>> wireshark-devel) from your operating system's package manager. >>>>> >>>>> Then, get the UHD source code, and >>>>> >>>>> cd /path/to/uhd/tools/dissectors >>>>> mkdir build >>>>> cd build >>>>> cmake -DETTUS_DISSECTOR_NAME=zpu .. >>>>> make -j4 && make install >>>>> >>>>> Ideally, add your user to the group that has access to the raw >>>>> network hardware (in case of my OS, that was the "wireshark" >>>>> group), log out and back in. >>>>> >>>>> If that doesn't work, you have to record as root and analyze as >>>>> your user (the USRP protocol dissectors got installed to your home >>>>> directory). >>>>> >>>>> Then, record the traffic on your 10G interface, and analyze the >>>>> packets in question as UHD (some/many will be sample packets, i.e. >>>>> CVITA). >>>>> >>>>> Best regards, >>>>> >>>>> Marcus >>>>> >>>>> On 24.05.2017 20:15, ROBIN TORTORA via USRP-users wrote: >>>>>> >>>>>> The only thing that jumps out to me is they are all on the same >>>>>> subnet... >>>>>> >>>>>> >>>>>> That is something I have typically avoided, but if you say it >>>>>> works in another configuration, that may not be the issue... >>>>>> >>>>>>> On May 24, 2017 at 12:12 PM Kevin Krieger via USRP-users >>>>>>> <usrp-users@lists.ettus.com> wrote: >>>>>>> >>>>>>> Hi all, >>>>>>> >>>>>>> I've got a predicament. Here's some *background* before I >>>>>>> describe my problem: >>>>>>> We purchased a Netgear xs728T 10GB switch in order to network 16 >>>>>>> N200 devices, as well as a processing computer (which has a 10G >>>>>>> ethernet card). Our 16 N200 devices are going to be used in a >>>>>>> radar configuration where they are all simultaneously either >>>>>>> transmitting or receiving. They are all receiving coincident >>>>>>> 10MHz and 1PPS signals from 2 octoclocks, which are both fed >>>>>>> from a third clock to ensure coincident clocks. They are >>>>>>> connected to the switch via cat6 cables ~7feet in length (this >>>>>>> item: >>>>>>> https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136) >>>>>>> >>>>>>> *The problem:* >>>>>>> During testing we've found that the example program tx_bursts, >>>>>>> when run on one USRP at a time with a modest bandwidth, the >>>>>>> "Waiting for async burst ack....failed" message appears about >>>>>>> 75% of the time. If I try to use two or more USRPs at a time, >>>>>>> then it appears every time. >>>>>>> The line used for tx_bursts was typically something like >>>>>>> this:./tx_bursts --args="addr0=192.168.10.104" --freq 10e6 >>>>>>> --rate=1e6 --channels="0" >>>>>>> for a one usrp test. For a multiple usrp test, I used lines like >>>>>>> this:./tx_bursts >>>>>>> --args="addr0=192.168.10.101,addr1=192.168.10.102,addr2=192.168.10.103,addr3=192.168.10.100,addr4=192.168.10.112,addr5=192.168.10.113,addr6=192.168.10.114" >>>>>>> --freq 10e6 --rate=1e6 --channels="0,1,2,3,4,5,6" >>>>>>> >>>>>>> >>>>>>> *Some evidence & information: >>>>>>> *We also have a 5 port 1G DLink DGS-1005G. We've tested 4 N200s >>>>>>> with tx_bursts on this switch (same cabling) and it works >>>>>>> flawlessly. >>>>>>> We also have tested a second 10G switch, the 8 port XS708E with >>>>>>> 7 N200s with tx_bursts and it works flawlessly. >>>>>>> >>>>>>> I have wireshark dumps taken via a third machine that is >>>>>>> connected to mirrored ports on the xs728T switch. I have >>>>>>> attached them in case anyone can tell me if they see something >>>>>>> wrong besides the missing acks. >>>>>>> The wireshark filter to use is : ip.src == 192.168.10.104 and >>>>>>> ip.dst == 192.168.10.104 or ip.src == 192.168.10.104 and ip.dst >>>>>>> == 192.168.10.99 >>>>>>> >>>>>>> The xs728t switch information is: Boot version is 1.0.0.0, SW >>>>>>> version is 6.5.1.26 (latest as of testing). >>>>>>> I contacted Netgear support, and they sent us a brand new >>>>>>> switch, which exhibited the same problem. >>>>>>> >>>>>>> I'm wondering if there's anyone out there who has had similar >>>>>>> issues? Is there anything I can do to get more information or >>>>>>> get around this problem? >>>>>>> Can anyone see what the root cause might be in the wireshark >>>>>>> captures? >>>>>>> >>>>>>> Any help or pointers are appreciated. Thank you, >>>>>>> Kevin >>>>>> >>>>>> >>>>>>> _______________________________________________ >>>>>>> USRP-users mailing list >>>>>>> USRP-users@lists.ettus.com >>>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> USRP-users mailing list >>>>>> USRP-users@lists.ettus.com >>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> USRP-users mailing list >>>>> USRP-users@lists.ettus.com >>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>>> >>>> >>>> >>>> _______________________________________________ >>>> USRP-users mailing list >>>> USRP-users@lists.ettus.com >>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>> >>> >>> >>> _______________________________________________ >>> USRP-users mailing list >>> USRP-users@lists.ettus.com >>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >> >> _______________________________________________ >> USRP-users mailing list >> USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> >> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >

Kevin, I wouldn’t expend more effort at the moment on the dissector, the UDP ports and packet sizes pretty much tell you whats going on if you understand UHD. It may be that the dissector for the old UHD protocol was never complete/perfect. Interesting that it was the monitoring port that was 10G and the host 1G….this kind of illustrates my point about using a dedicated non rate converting monitoring switch since the 10G port delivered bursts of packets to Wireshark at intervals that were much too close to have been the original 1G timing. Maybe your route forward is two XS708E’s if that’s robust…frustrating to not solve the issue but a better use of your time I suspect. -Ian > On Nov 10, 2017, at 9:17 AM, Kevin Krieger <kevin.krieger@usask.ca> wrote: > > Hi Ian, > > OK - is there code for the dissector available to build a new version? Or do you think it's worth it? > > Also - our host sending the samples was actually only 1Gbps ethernet (we weren't using the 10Gbps card for that), so I'm not sure that the samples would be going into the switch faster than this. We were using the 10G card for wireshark. > Thanks for the info about the N210 ethernet port, maybe we can try forcing the switch ports to 1G and see if that helps. > Also adding a 1G switch in between the USRP and 10G will be a good test if we can get our hands on a 1G switch. > We discussed just getting four 4port 1G cards, we would need to upgrade our motherboard though. > > Thanks, If we figure anything out we'll post back here. > > Regards, > Kevin > > On 11/09/2017 09:14 AM, Ian Buckley wrote: >> >> Kevin, >> I glanced through Captures 0,1,2 and could see the general gist of where it all goes wrong, but it’s not really clear why. The dissector seems out of date, at least it isn’t decoding correctly but I can still follow it. >> >> Capture 0: >> Packets 2551 through 2606 are the actual sample data packets. >> Then there is an approx 1.2second pause before UHD, having timed out waiting for ACK’s begins some basic control interaction initiated by the host. >> >> Contrast that with Capture 2, where you will see a nicely paced stream of ACK’s (starts at packet 2609) flowing back from the USRP at exact intervals pacing the real time TX operation as the DSP consumes the samples. >> >> Capture 1: >> This goes to the weeds early, streaming never gets close to starting. At around packet 1324 you suddenly see a series of pauses for exactly 0.5 secs each. >> >> So a couple of thoughts bordering on guesses: >> >> Rate conversion in switches is always a little bit of a minefield when you have a single stream that is faster than the skinny pipe. In this case, operations like the sample burst are streaming from the host faster than 1Gbs into the switch forcing the switch to buffer samples. It’s surprising sometimes how quick these lower cost single chip switches can get buffer starved. Try using 'ethtool' on the Linux host to force the host port to 1Gbps to eliminate rate conversion in the switch as a cause. >> >> N210 in general is the most reliable of all USRP’s but it has one quirk, which is that the ethernet port is forced to be 1Gbps and it does no auto negotiation. This often trips up people who connect it to a 100mbit switch or host. I have never actually tried connecting it to 10GBase-T switches…there may be some (switch implementation specific) gotcha’s. >> >> Mirroring multiple ports to a single port can affect how the problem manifests because you create new bottle necks in the switch …if the 10G switch is suspect then I suggest interposing a cheap 1G switch that supports mirroring between USRP and 10G switch, that way you’ll have higher confidence exactly what and when went onto the wire. This is also an interesting experiment because it isolates the N210’s ethernet port from the 10G switch…curious to see if it now starts to work OK. I have had this happen when debugging other products. >> >> The ICMP port unreachable is likely a red herring, some spurious Linux service rather than UHD originating those packets I suspect. >> >> Might be worth looking at the switch internal counters to see if any physical layer error counters are increasing. >> >> No smoking gun I’m afraid; your fall back plan might be to stuff quad port 1G cards in your host, they’re cheap if you have the PCIe slots free. >> >> -Ian >> >>> On Nov 6, 2017, at 3:13 PM, Kevin Krieger via USRP-users <usrp-users@lists.ettus.com <mailto:usrp-users@lists.ettus.com>> wrote: >>> >>> Hi, >>> >>> Any chance to look at these wireshark captures? >>> >>> Thanks, >>> Kevin >>> >>> On 10/30/2017 05:30 PM, Kevin Krieger via USRP-users wrote: >>>> Hi Marcus, >>>> >>>> I finally got around to this. >>>> I've attached 9 captures. >>>> >>>> The setup for the captures was: >>>> XS728T with latest firmware running, factory default >>>> Computer running tx_bursts has IP of 192.168.10.60 or .67 >>>> N200 has IP of 192.168.10.107 >>>> ./tx_bursts --args addr0=192.168.10.107 --freq 10e6 --rate 1e6 >>>> >>>> Captures 0,1 and 2 are with both the ports (n200 side, and computer side) mirrored to the wireshark capture port, so you essentially see duplicate traffic. >>>> Captures 3,4 and 5 are with only the computer side traffic captured >>>> Captures 6,7 and 8 are with only the N200 side traffic captured. >>>> >>>> Captures 0, 4, and 8 are with the messages "Sent packet: 363 samples" appearing, but then the "Waiting for async burst ACK ... fail" message also appears. >>>> Captures 1, 3 and 7 are with the message :Error: Runtimeerror: fifo ctrl timed out looking for acks" appearing. >>>> Captures 2, 5 and 6 are with successful runs of the tx bursts program, so instead of 'fail' at the end there is 'success'. >>>> >>>> There are txt files with the output of the program for each capture, as well as csv files showing the dissected output (only found uhd in the dissectors). >>>> >>>> Can you see anything wrong with these runs? Any help is appreciated. >>>> >>>> Thanks, >>>> Kevin >>>> >>>> >>>> On 06/15/2017 02:28 PM, Kevin Krieger via USRP-users wrote: >>>>> Hi Marcus, >>>>> >>>>> Thank you - I was sure that the same subnet was correct but it's good to have confirmation. >>>>> I have done the dissecting previously, and I recall getting a bunch of "huh what's that?"' messages and then a 'wassup bro' or something but I'll do it again and let you know what I see. (Hilarious descriptions btw!) >>>>> >>>>> Kevin >>>>> >>>>> >>>>> On 05/24/2017 08:07 PM, Marcus Müller via USRP-users wrote: >>>>>> Hi Kevin, Hi Robin, >>>>>> the same-subnet configuration is the right one, here. There's no routing ambiguity if you have one card that serves the whole switch. >>>>>> Kevin, this might be much to ask, but: I'd ask you to both install wireshark and its development headers (typically, wireshark-dev or wireshark-devel) from your operating system's package manager. >>>>>> Then, get the UHD source code, and >>>>>> >>>>>> cd /path/to/uhd/tools/dissectors >>>>>> mkdir build >>>>>> cd build >>>>>> cmake -DETTUS_DISSECTOR_NAME=zpu .. >>>>>> make -j4 && make install >>>>>> >>>>>> Ideally, add your user to the group that has access to the raw network hardware (in case of my OS, that was the "wireshark" group), log out and back in. >>>>>> >>>>>> If that doesn't work, you have to record as root and analyze as your user (the USRP protocol dissectors got installed to your home directory). >>>>>> >>>>>> Then, record the traffic on your 10G interface, and analyze the packets in question as UHD (some/many will be sample packets, i.e. CVITA). >>>>>> >>>>>> Best regards, >>>>>> >>>>>> Marcus >>>>>> On 24.05.2017 20:15, ROBIN TORTORA via USRP-users wrote: >>>>>>> The only thing that jumps out to me is they are all on the same subnet... >>>>>>> >>>>>>> >>>>>>> That is something I have typically avoided, but if you say it works in another configuration, that may not be the issue... >>>>>>> >>>>>>>> On May 24, 2017 at 12:12 PM Kevin Krieger via USRP-users <usrp-users@lists.ettus.com> <mailto:usrp-users@lists.ettus.com> wrote: >>>>>>>> >>>>>>>> Hi all, >>>>>>>> >>>>>>>> I've got a predicament. Here's some background before I describe my problem: >>>>>>>> We purchased a Netgear xs728T 10GB switch in order to network 16 N200 devices, as well as a processing computer (which has a 10G ethernet card). Our 16 N200 devices are going to be used in a radar configuration where they are all simultaneously either transmitting or receiving. They are all receiving coincident 10MHz and 1PPS signals from 2 octoclocks, which are both fed from a third clock to ensure coincident clocks. They are connected to the switch via cat6 cables ~7feet in length (this item: https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136 <https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136>) >>>>>>>> >>>>>>>> The problem: >>>>>>>> During testing we've found that the example program tx_bursts, when run on one USRP at a time with a modest bandwidth, the "Waiting for async burst ack....failed" message appears about 75% of the time. If I try to use two or more USRPs at a time, then it appears every time. >>>>>>>> The line used for tx_bursts was typically something like this: ./tx_bursts --args="addr0=192.168.10.104" --freq 10e6 --rate=1e6 --channels="0" >>>>>>>> for a one usrp test. For a multiple usrp test, I used lines like this: ./tx_bursts --args="addr0=192.168.10.101,addr1=192.168.10.102,addr2=192.168.10.103,addr3=192.168.10.100,addr4=192.168.10.112,addr5=192.168.10.113,addr6=192.168.10.114" --freq 10e6 --rate=1e6 --channels="0,1,2,3,4,5,6" >>>>>>>> >>>>>>>> >>>>>>>> Some evidence & information: >>>>>>>> We also have a 5 port 1G DLink DGS-1005G. We've tested 4 N200s with tx_bursts on this switch (same cabling) and it works flawlessly. >>>>>>>> We also have tested a second 10G switch, the 8 port XS708E with 7 N200s with tx_bursts and it works flawlessly. >>>>>>>> >>>>>>>> I have wireshark dumps taken via a third machine that is connected to mirrored ports on the xs728T switch. I have attached them in case anyone can tell me if they see something wrong besides the missing acks. >>>>>>>> The wireshark filter to use is : ip.src == 192.168.10.104 and ip.dst == 192.168.10.104 or ip.src == 192.168.10.104 and ip.dst == 192.168.10.99 >>>>>>>> >>>>>>>> The xs728t switch information is: Boot version is 1.0.0.0, SW version is 6.5.1.26 (latest as of testing). >>>>>>>> I contacted Netgear support, and they sent us a brand new switch, which exhibited the same problem. >>>>>>>> >>>>>>>> I'm wondering if there's anyone out there who has had similar issues? Is there anything I can do to get more information or get around this problem? >>>>>>>> Can anyone see what the root cause might be in the wireshark captures? >>>>>>>> >>>>>>>> Any help or pointers are appreciated. Thank you, >>>>>>>> Kevin >>>>>>> >>>>>>> >>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> USRP-users mailing list >>>>>>>> USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> >>>>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> USRP-users mailing list >>>>>>> USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> >>>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com> >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> USRP-users mailing list >>>>>> USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> >>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> USRP-users mailing list >>>>> USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> >>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com> >>>> >>>> >>>> >>>> _______________________________________________ >>>> USRP-users mailing list >>>> USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> >>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com> >>> >>> _______________________________________________ >>> USRP-users mailing list >>> USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> >>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com <http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com> >> >

Hi Ian, Just as an update, we've purchased 3 XS708E's. We've daisy chained them and we have all 16 N200 devices plugged in, a 10G intel card in the computer, and the tx_bursts example runs flawlessly on all the devices at once. It's cheaper than buying one XS728T, but it takes up more space and I suspect daisy-chaining 3 switches together isn't best practice... NETGEAR is currently looking into the issue with why the XS728T doesn't work, so hopefully they figure that out. In the meantime, we move on with 3 switches. Kevin On 11/10/2017 05:52 PM, Ian Buckley wrote: > Kevin, > I wouldn’t expend more effort at the moment on the dissector, the UDP > ports and packet sizes pretty much tell you whats going on if you > understand UHD. It may be that the dissector for the old UHD protocol > was never complete/perfect. > > Interesting that it was the monitoring port that was 10G and the host > 1G….this kind of illustrates my point about using a dedicated non rate > converting monitoring switch since the 10G port delivered bursts of > packets to Wireshark at intervals that were much too close to have > been the original 1G timing. > > Maybe your route forward is two XS708E’s if that’s robust…frustrating > to not solve the issue but a better use of your time I suspect. > -Ian > > >> On Nov 10, 2017, at 9:17 AM, Kevin Krieger <kevin.krieger@usask.ca >> <mailto:kevin.krieger@usask.ca>> wrote: >> >> Hi Ian, >> >> OK - is there code for the dissector available to build a new >> version? Or do you think it's worth it? >> >> Also - our host sending the samples was actually only 1Gbps ethernet >> (we weren't using the 10Gbps card for that), so I'm not sure that the >> samples would be going into the switch faster than this. We were >> using the 10G card for wireshark. >> Thanks for the info about the N210 ethernet port, maybe we can try >> forcing the switch ports to 1G and see if that helps. >> Also adding a 1G switch in between the USRP and 10G will be a good >> test if we can get our hands on a 1G switch. >> We discussed just getting four 4port 1G cards, we would need to >> upgrade our motherboard though. >> >> Thanks, If we figure anything out we'll post back here. >> >> Regards, >> Kevin >> >> On 11/09/2017 09:14 AM, Ian Buckley wrote: >>> >>> Kevin, >>> I glanced through Captures 0,1,2 and could see the general gist of >>> where it all goes wrong, but it’s not really clear why. The >>> dissector seems out of date, at least it isn’t decoding correctly >>> but I can still follow it. >>> >>> Capture 0: >>> Packets 2551 through 2606 are the actual sample data packets. >>> Then there is an approx 1.2second pause before UHD, having timed out >>> waiting for ACK’s begins some basic control interaction initiated by >>> the host. >>> >>> Contrast that with Capture 2, where you will see a nicely paced >>> stream of ACK’s (starts at packet 2609) flowing back from the USRP >>> at exact intervals pacing the real time TX operation as the DSP >>> consumes the samples. >>> >>> Capture 1: >>> This goes to the weeds early, streaming never gets close to >>> starting. At around packet 1324 you suddenly see a series of pauses >>> for exactly 0.5 secs each. >>> >>> So a couple of thoughts bordering on guesses: >>> Rate conversion in switches is always a little bit of a minefield >>> when you have a single stream that is faster than the skinny pipe. >>> In this case, operations like the sample burst are streaming from >>> the host faster than 1Gbs into the switch forcing the switch to >>> buffer samples. It’s surprising sometimes how quick these lower cost >>> single chip switches can get buffer starved. Try using 'ethtool' on >>> the Linux host to force the host port to 1Gbps to eliminate rate >>> conversion in the switch as a cause. >>> >>> N210 in general is the most reliable of all USRP’s but it has one >>> quirk, which is that the ethernet port is forced to be 1Gbps and it >>> does no auto negotiation. This often trips up people who connect it >>> to a 100mbit switch or host. I have never actually tried connecting >>> it to 10GBase-T switches…there may be some (switch implementation >>> specific) gotcha’s. >>> >>> Mirroring multiple ports to a single port can affect how the problem >>> manifests because you create new bottle necks in the switch …if the >>> 10G switch is suspect then I suggest interposing a cheap 1G switch >>> that supports mirroring between USRP and 10G switch, that way you’ll >>> have higher confidence exactly what and when went onto the wire. >>> This is also an interesting experiment because it isolates the >>> N210’s ethernet port from the 10G switch…curious to see if it now >>> starts to work OK. I have had this happen when debugging other products. >>> >>> The ICMP port unreachable is likely a red herring, some spurious >>> Linux service rather than UHD originating those packets I suspect. >>> >>> Might be worth looking at the switch internal counters to see if any >>> physical layer error counters are increasing. >>> >>> No smoking gun I’m afraid; your fall back plan might be to stuff >>> quad port 1G cards in your host, they’re cheap if you have the PCIe >>> slots free. >>> >>> -Ian >>> >>>> On Nov 6, 2017, at 3:13 PM, Kevin Krieger via USRP-users >>>> <usrp-users@lists.ettus.com <mailto:usrp-users@lists.ettus.com>> wrote: >>>> >>>> Hi, >>>> >>>> Any chance to look at these wireshark captures? >>>> >>>> Thanks, >>>> Kevin >>>> >>>> On 10/30/2017 05:30 PM, Kevin Krieger via USRP-users wrote: >>>>> Hi Marcus, >>>>> >>>>> I finally got around to this. >>>>> I've attached 9 captures. >>>>> >>>>> The setup for the captures was: >>>>> XS728T with latest firmware running, factory default >>>>> Computer running tx_bursts has IP of 192.168.10.60 or .67 >>>>> N200 has IP of 192.168.10.107 >>>>> ./tx_bursts --args addr0=192.168.10.107 --freq 10e6 --rate 1e6 >>>>> >>>>> Captures 0,1 and 2 are with both the ports (n200 side, and >>>>> computer side) mirrored to the wireshark capture port, so you >>>>> essentially see duplicate traffic. >>>>> Captures 3,4 and 5 are with only the computer side traffic captured >>>>> Captures 6,7 and 8 are with only the N200 side traffic captured. >>>>> >>>>> Captures 0, 4, and 8 are with the messages "Sent packet: 363 >>>>> samples" appearing, but then the "Waiting for async burst ACK ... >>>>> fail" message also appears. >>>>> Captures 1, 3 and 7 are with the message :Error: Runtimeerror: >>>>> fifo ctrl timed out looking for acks" appearing. >>>>> Captures 2, 5 and 6 are with successful runs of the tx bursts >>>>> program, so instead of 'fail' at the end there is 'success'. >>>>> >>>>> There are txt files with the output of the program for each >>>>> capture, as well as csv files showing the dissected output (only >>>>> found uhd in the dissectors). >>>>> >>>>> Can you see anything wrong with these runs? Any help is appreciated. >>>>> >>>>> Thanks, >>>>> Kevin >>>>> >>>>> >>>>> On 06/15/2017 02:28 PM, Kevin Krieger via USRP-users wrote: >>>>>> Hi Marcus, >>>>>> >>>>>> Thank you - I was sure that the same subnet was correct but it's >>>>>> good to have confirmation. >>>>>> I have done the dissecting previously, and I recall getting a >>>>>> bunch of "huh what's that?"' messages and then a 'wassup bro' or >>>>>> something but I'll do it again and let you know what I see. >>>>>> (Hilarious descriptions btw!) >>>>>> >>>>>> Kevin >>>>>> >>>>>> >>>>>> On 05/24/2017 08:07 PM, Marcus Müller via USRP-users wrote: >>>>>>> >>>>>>> Hi Kevin, Hi Robin, >>>>>>> >>>>>>> the same-subnet configuration is the right one, here. There's no >>>>>>> routing ambiguity if you have one card that serves the whole >>>>>>> switch. >>>>>>> >>>>>>> Kevin, this might be much to ask, but: I'd ask you to both >>>>>>> install wireshark and its development headers (typically, >>>>>>> wireshark-dev or wireshark-devel) from your operating system's >>>>>>> package manager. >>>>>>> >>>>>>> Then, get the UHD source code, and >>>>>>> >>>>>>> cd /path/to/uhd/tools/dissectors >>>>>>> mkdir build >>>>>>> cd build >>>>>>> cmake -DETTUS_DISSECTOR_NAME=zpu .. >>>>>>> make -j4 && make install >>>>>>> >>>>>>> Ideally, add your user to the group that has access to the raw >>>>>>> network hardware (in case of my OS, that was the "wireshark" >>>>>>> group), log out and back in. >>>>>>> >>>>>>> If that doesn't work, you have to record as root and analyze as >>>>>>> your user (the USRP protocol dissectors got installed to your >>>>>>> home directory). >>>>>>> >>>>>>> Then, record the traffic on your 10G interface, and analyze the >>>>>>> packets in question as UHD (some/many will be sample packets, >>>>>>> i.e. CVITA). >>>>>>> >>>>>>> Best regards, >>>>>>> >>>>>>> Marcus >>>>>>> >>>>>>> On 24.05.2017 20:15, ROBIN TORTORA via USRP-users wrote: >>>>>>>> >>>>>>>> The only thing that jumps out to me is they are all on the same >>>>>>>> subnet... >>>>>>>> >>>>>>>> >>>>>>>> That is something I have typically avoided, but if you say it >>>>>>>> works in another configuration, that may not be the issue... >>>>>>>> >>>>>>>>> On May 24, 2017 at 12:12 PM Kevin Krieger via USRP-users >>>>>>>>> <usrp-users@lists.ettus.com> wrote: >>>>>>>>> >>>>>>>>> Hi all, >>>>>>>>> >>>>>>>>> I've got a predicament. Here's some *background* before I >>>>>>>>> describe my problem: >>>>>>>>> We purchased a Netgear xs728T 10GB switch in order to network >>>>>>>>> 16 N200 devices, as well as a processing computer (which has a >>>>>>>>> 10G ethernet card). Our 16 N200 devices are going to be used >>>>>>>>> in a radar configuration where they are all simultaneously >>>>>>>>> either transmitting or receiving. They are all receiving >>>>>>>>> coincident 10MHz and 1PPS signals from 2 octoclocks, which are >>>>>>>>> both fed from a third clock to ensure coincident clocks. They >>>>>>>>> are connected to the switch via cat6 cables ~7feet in length >>>>>>>>> (this item: >>>>>>>>> https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136) >>>>>>>>> >>>>>>>>> *The problem:* >>>>>>>>> During testing we've found that the example program tx_bursts, >>>>>>>>> when run on one USRP at a time with a modest bandwidth, the >>>>>>>>> "Waiting for async burst ack....failed" message appears about >>>>>>>>> 75% of the time. If I try to use two or more USRPs at a time, >>>>>>>>> then it appears every time. >>>>>>>>> The line used for tx_bursts was typically something like >>>>>>>>> this:./tx_bursts --args="addr0=192.168.10.104" --freq 10e6 >>>>>>>>> --rate=1e6 --channels="0" >>>>>>>>> for a one usrp test. For a multiple usrp test, I used lines >>>>>>>>> like this:./tx_bursts >>>>>>>>> --args="addr0=192.168.10.101,addr1=192.168.10.102,addr2=192.168.10.103,addr3=192.168.10.100,addr4=192.168.10.112,addr5=192.168.10.113,addr6=192.168.10.114" >>>>>>>>> --freq 10e6 --rate=1e6 --channels="0,1,2,3,4,5,6" >>>>>>>>> >>>>>>>>> >>>>>>>>> *Some evidence & information: >>>>>>>>> *We also have a 5 port 1G DLink DGS-1005G. We've tested 4 >>>>>>>>> N200s with tx_bursts on this switch (same cabling) and it >>>>>>>>> works flawlessly. >>>>>>>>> We also have tested a second 10G switch, the 8 port XS708E >>>>>>>>> with 7 N200s with tx_bursts and it works flawlessly. >>>>>>>>> >>>>>>>>> I have wireshark dumps taken via a third machine that is >>>>>>>>> connected to mirrored ports on the xs728T switch. I have >>>>>>>>> attached them in case anyone can tell me if they see something >>>>>>>>> wrong besides the missing acks. >>>>>>>>> The wireshark filter to use is : ip.src == 192.168.10.104 and >>>>>>>>> ip.dst == 192.168.10.104 or ip.src == 192.168.10.104 and >>>>>>>>> ip.dst == 192.168.10.99 >>>>>>>>> >>>>>>>>> The xs728t switch information is: Boot version is 1.0.0.0, SW >>>>>>>>> version is 6.5.1.26 (latest as of testing). >>>>>>>>> I contacted Netgear support, and they sent us a brand new >>>>>>>>> switch, which exhibited the same problem. >>>>>>>>> >>>>>>>>> I'm wondering if there's anyone out there who has had similar >>>>>>>>> issues? Is there anything I can do to get more information or >>>>>>>>> get around this problem? >>>>>>>>> Can anyone see what the root cause might be in the wireshark >>>>>>>>> captures? >>>>>>>>> >>>>>>>>> Any help or pointers are appreciated. Thank you, >>>>>>>>> Kevin >>>>>>>> >>>>>>>> >>>>>>>>> _______________________________________________ >>>>>>>>> USRP-users mailing list >>>>>>>>> USRP-users@lists.ettus.com >>>>>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> USRP-users mailing list >>>>>>>> USRP-users@lists.ettus.com >>>>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>>>>>> >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> USRP-users mailing list >>>>>>> USRP-users@lists.ettus.com >>>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>>>>> >>>>>> >>>>>> >>>>>> _______________________________________________ >>>>>> USRP-users mailing list >>>>>> USRP-users@lists.ettus.com >>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> USRP-users mailing list >>>>> USRP-users@lists.ettus.com >>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>>> >>>> _______________________________________________ >>>> USRP-users mailing list >>>> USRP-users@lists.ettus.com <mailto:USRP-users@lists.ettus.com> >>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com >>> >> >