[USRP-users] Netgear 10GB switch XS728T

Kevin Krieger kevin.krieger at usask.ca
Fri Nov 10 12:17:42 EST 2017

Hi Ian,

OK - is there code for the dissector available to build a new version? 
Or do you think it's worth it?

Also - our host sending the samples was actually only 1Gbps ethernet (we 
weren't using the 10Gbps card for that), so I'm not sure that the 
samples would be going into the switch faster than this. We were using 
the 10G card for wireshark.
Thanks for the info about the N210 ethernet port, maybe we can try 
forcing the switch ports to 1G and see if that helps.
Also adding a 1G switch in between the USRP and 10G will be a good test 
if we can get our hands on a 1G switch.
We discussed just getting four 4port 1G cards, we would need to upgrade 
our motherboard though.

Thanks, If we figure anything out we'll post back here.


On 11/09/2017 09:14 AM, Ian Buckley wrote:
> Kevin,
> I glanced through Captures 0,1,2 and could see the general gist of 
> where it all goes wrong, but it’s not really clear why. The dissector 
> seems out of date, at least it isn’t decoding correctly but I can 
> still follow it.
> Capture 0:
> Packets 2551 through 2606 are the actual sample data packets.
> Then there is an approx 1.2second pause before UHD, having timed out 
> waiting for ACK’s begins some basic control interaction initiated by 
> the host.
> Contrast that with Capture 2, where you will see a nicely paced stream 
> of ACK’s (starts at packet 2609) flowing back from the USRP at exact 
> intervals pacing the real time TX operation as the DSP consumes the 
> samples.
> Capture 1:
> This goes to the weeds early, streaming never gets close to starting. 
> At around packet 1324 you suddenly see a series of pauses for exactly 
> 0.5 secs each.
> So a couple of thoughts bordering on guesses:
> Rate conversion in switches is always a little bit of a minefield when 
> you have a single stream that is faster than the skinny pipe. In this 
> case, operations like the sample burst are streaming from the host 
> faster than 1Gbs into the switch forcing the switch to buffer samples. 
> It’s surprising sometimes how quick these lower cost single chip 
> switches can get buffer starved. Try using 'ethtool' on the Linux host 
> to force the host port to 1Gbps to eliminate rate conversion in the 
> switch as a cause.
> N210 in general is the most reliable of all USRP’s but it has one 
> quirk, which is that the ethernet port is forced to be 1Gbps and it 
> does no auto negotiation. This often trips up people who connect it to 
> a 100mbit switch or host. I have never actually tried connecting it to 
> 10GBase-T switches…there may be some (switch implementation specific) 
> gotcha’s.
> Mirroring multiple ports to a single port can affect how the problem 
> manifests because you create new bottle necks in the switch …if the 
> 10G switch is suspect then I suggest interposing a cheap 1G switch 
> that supports mirroring between USRP and 10G switch, that way you’ll 
> have higher confidence exactly what and when went onto the wire. This 
> is also an interesting experiment because it isolates the N210’s 
> ethernet port from the 10G switch…curious to see if it now starts to 
> work OK. I have had this happen when debugging other products.
> The ICMP port unreachable is likely a red herring, some spurious Linux 
> service rather than UHD originating those packets I suspect.
> Might be worth looking at the switch internal counters to see if any 
> physical layer error counters are increasing.
> No smoking gun I’m afraid; your fall back plan might be to stuff quad 
> port 1G cards in your host, they’re cheap if you have the PCIe slots free.
> -Ian
>> On Nov 6, 2017, at 3:13 PM, Kevin Krieger via USRP-users 
>> <usrp-users at lists.ettus.com <mailto:usrp-users at lists.ettus.com>> wrote:
>> Hi,
>> Any chance to look at these wireshark captures?
>> Thanks,
>> Kevin
>> On 10/30/2017 05:30 PM, Kevin Krieger via USRP-users wrote:
>>> Hi Marcus,
>>> I finally got around to this.
>>> I've attached 9 captures.
>>> The setup for the captures was:
>>> XS728T with latest firmware running, factory default
>>> Computer running tx_bursts has IP of or .67
>>> N200 has IP of
>>>  ./tx_bursts  --args addr0=  --freq 10e6 --rate 1e6
>>> Captures 0,1 and 2 are with both the ports (n200 side, and computer 
>>> side) mirrored to the wireshark capture port, so you essentially see 
>>> duplicate traffic.
>>> Captures 3,4 and 5 are with only the computer side traffic captured
>>> Captures 6,7 and 8 are with only the N200 side traffic captured.
>>> Captures 0, 4, and 8 are with the messages "Sent packet: 363 
>>> samples" appearing, but then the "Waiting for async burst ACK ... 
>>> fail" message also appears.
>>> Captures 1, 3 and 7 are with the message :Error: Runtimeerror: fifo 
>>> ctrl timed out looking for acks" appearing.
>>> Captures 2, 5 and 6 are with successful runs of the tx bursts 
>>> program, so instead of 'fail' at the end there is 'success'.
>>> There are txt files with the output of the program for each capture, 
>>> as well as csv files showing the dissected output (only found uhd in 
>>> the dissectors).
>>> Can you see anything wrong with these runs? Any help is appreciated.
>>> Thanks,
>>> Kevin
>>> On 06/15/2017 02:28 PM, Kevin Krieger via USRP-users wrote:
>>>> Hi Marcus,
>>>> Thank you - I was sure that the same subnet was correct but it's 
>>>> good to have confirmation.
>>>> I have done the dissecting previously, and I recall getting a bunch 
>>>> of "huh what's that?"' messages and then a 'wassup bro' or 
>>>> something but I'll do it again and let you know what I see. 
>>>> (Hilarious descriptions btw!)
>>>> Kevin
>>>> On 05/24/2017 08:07 PM, Marcus Müller via USRP-users wrote:
>>>>> Hi Kevin, Hi Robin,
>>>>> the same-subnet configuration is the right one, here. There's no 
>>>>> routing ambiguity if you have one card that serves the whole switch.
>>>>> Kevin, this might be much to ask, but: I'd ask you to both install 
>>>>> wireshark and its development headers (typically, wireshark-dev or 
>>>>> wireshark-devel) from your operating system's package manager.
>>>>> Then, get the UHD source code, and
>>>>> cd /path/to/uhd/tools/dissectors
>>>>> mkdir build
>>>>> cd build
>>>>> cmake -DETTUS_DISSECTOR_NAME=zpu ..
>>>>> make -j4 && make install
>>>>> Ideally, add your user to the group that has access to the raw 
>>>>> network hardware (in case of my OS, that was the "wireshark" 
>>>>> group), log out and back in.
>>>>> If that doesn't work, you have to record as root and analyze as 
>>>>> your user (the USRP protocol dissectors got installed to your home 
>>>>> directory).
>>>>> Then, record the traffic on your 10G interface, and analyze the 
>>>>> packets in question as UHD (some/many will be sample packets, i.e. 
>>>>> CVITA).
>>>>> Best regards,
>>>>> Marcus
>>>>> On 24.05.2017 20:15, ROBIN TORTORA via USRP-users wrote:
>>>>>> The only thing that jumps out to me is they are all on the same 
>>>>>> subnet...
>>>>>> That is something I have typically avoided, but if you say it 
>>>>>> works in another configuration, that may not be the issue...
>>>>>>> On May 24, 2017 at 12:12 PM Kevin Krieger via USRP-users 
>>>>>>> <usrp-users at lists.ettus.com> wrote:
>>>>>>> Hi all,
>>>>>>> I've got a predicament. Here's some *background* before I 
>>>>>>> describe my problem:
>>>>>>> We purchased a Netgear xs728T 10GB switch in order to network 16 
>>>>>>> N200 devices, as well as a processing computer (which has a 10G 
>>>>>>> ethernet card). Our 16 N200 devices are going to be used in a 
>>>>>>> radar configuration where they are all simultaneously either 
>>>>>>> transmitting or receiving. They are all receiving coincident 
>>>>>>> 10MHz and 1PPS signals from 2 octoclocks, which are both fed 
>>>>>>> from a third clock to ensure coincident clocks. They are 
>>>>>>> connected to the switch via cat6 cables ~7feet in length (this 
>>>>>>> item: 
>>>>>>> https://www.newegg.ca/Product/Product.aspx?Item=N82E16812119168&_ga=2.227605844.817843194.1495640587-1295759170.1493911136)
>>>>>>> *The problem:*
>>>>>>> During testing we've found that the example program tx_bursts, 
>>>>>>> when run on one USRP at a time with a modest bandwidth, the 
>>>>>>> "Waiting for async burst ack....failed" message appears about 
>>>>>>> 75% of the time. If I try to use two or more USRPs at a time, 
>>>>>>> then it appears every time.
>>>>>>> The line used for tx_bursts was typically something like 
>>>>>>> this:./tx_bursts --args="addr0=" --freq 10e6 
>>>>>>> --rate=1e6 --channels="0"
>>>>>>> for a one usrp test. For a multiple usrp test, I used lines like 
>>>>>>> this:./tx_bursts 
>>>>>>> --args="addr0=,addr1=,addr2=,addr3=,addr4=,addr5=,addr6=" 
>>>>>>> --freq 10e6 --rate=1e6 --channels="0,1,2,3,4,5,6"
>>>>>>> *Some evidence & information:
>>>>>>> *We also have a 5 port 1G DLink DGS-1005G. We've tested 4 N200s 
>>>>>>> with tx_bursts on this switch (same cabling) and it works 
>>>>>>> flawlessly.
>>>>>>> We also have tested a second 10G switch, the 8 port XS708E with 
>>>>>>> 7 N200s with tx_bursts and it works flawlessly.
>>>>>>> I have wireshark dumps taken via a third machine that is 
>>>>>>> connected to mirrored ports on the xs728T switch. I have 
>>>>>>> attached them in case anyone can tell me if they see something 
>>>>>>> wrong besides the missing acks.
>>>>>>> The wireshark filter to use is : ip.src == and 
>>>>>>> ip.dst == or ip.src == and ip.dst 
>>>>>>> ==
>>>>>>> The xs728t switch information is: Boot version is, SW 
>>>>>>> version is (latest as of testing).
>>>>>>> I contacted Netgear support, and they sent us a brand new 
>>>>>>> switch, which exhibited the same problem.
>>>>>>> I'm wondering if there's anyone out there who has had similar 
>>>>>>> issues? Is there anything I can do to get more information or 
>>>>>>> get around this problem?
>>>>>>> Can anyone see what the root cause might be in the wireshark 
>>>>>>> captures?
>>>>>>> Any help or pointers are appreciated. Thank you,
>>>>>>> Kevin
>>>>>>> _______________________________________________
>>>>>>> USRP-users mailing list
>>>>>>> USRP-users at lists.ettus.com
>>>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
>>>>>> _______________________________________________
>>>>>> USRP-users mailing list
>>>>>> USRP-users at lists.ettus.com
>>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
>>>>> _______________________________________________
>>>>> USRP-users mailing list
>>>>> USRP-users at lists.ettus.com
>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
>>>> _______________________________________________
>>>> USRP-users mailing list
>>>> USRP-users at lists.ettus.com
>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
>>> _______________________________________________
>>> USRP-users mailing list
>>> USRP-users at lists.ettus.com
>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
>> _______________________________________________
>> USRP-users mailing list
>> USRP-users at lists.ettus.com <mailto:USRP-users at lists.ettus.com>
>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/attachments/20171110/a341d798/attachment-0002.html>

More information about the USRP-users mailing list