[USRP-users] Recording the full X3x0 bandwidth
k5so at k5so.com
Sat Mar 9 11:30:34 EST 2019
Thanks for your post. I am new to USRP devices and UHD and GNURadio but I have recently obtained an X310 with dual TwinRX daughterboards to do pulsar radio astronomy and I have been exploring exactly the issue you raise in order to help me with my requirements to record data for hours at a time.
Thus far, my results are similar to your results it seems as I am able to write dual-channel 32-bit complex samples from the X310 to disk at 25Msps without loss of any samples, or at 50Msps using a single channel for hours at a time.
I’m using a single 10G ethernet connection of a dual-channel 10G interface to the PC. The PC is an AMD Threadripper 2950x CPU @3.4GHz with an Intel 2TB SSD and an 8TB HDD for storage. To achieve the above throughput I am writing directly to the SSD only as the HDD can’t keep up with these data rates.
At the moment I am using a GNURadio-Companion flowchart program to control the X310 but I will shortly be exploring use of my own C++ program utilizing only UHD routines to perform the transfer task if I can arrange that. The effort to use my own program is driven by my desire to explore using 16-bit complex or even 8-bit sample sizes from the X310 to allow me to record 100Msps from dual channels but I don’t think GRC allows those choices of sample format into the File Sink block, unless I’m misunderstanding the capabilities of GRC, so I’ll attempt to write my own in C++ using UHD to do this instead.
Being new to much of this I don’t know if I can be of any help to you but I thought I’d let you know that I for one am very interested in hearing of your progress as your and my goals appear to be similar with respect to achieving dual-channel, wide-bandwidth output from the X310 to disk.
> On Mar 9, 2019, at 4:32 AM, Paul Boven via USRP-users <usrp-users at lists.ettus.com> wrote:
> I'm trying to record the full X310 bandwidth, for a few hours, without any missed samples. Which of course is a bit of a challenge - does anyone here already achieve this?
> We're using a TwinRX, so initially I wanted to record 2x 100MS/s (from both channels), which amounts to 800MB/s, 6.4Gb/s. At first I tried uhd_rx_cfile, but have been unable to get it to a good state without showing an 'O' every few seconds at these speeds.
> As a recorder I have a SuperMicro 847 chassis with 36 disks (Seagate Ironwolf 8TB T8000VN0022, 7200rpm). In this particular server, the disks are connected through an 'expander' backplane, from a single HBA (LSI 3008). CPU is dual Xeon 4110, 2.1 GHz, 64 GB of ram.
> At first I tried a 6 disk pool (raidz1), and eventually ended up creating a huge 36 disk ZFS stripe, which in theory should have no trouble with the throughput, but certainly kept dropping packets.
> Note that recording to /dev/shm/file works perfectly without dropping packets, until the point that the memory is full.
> Given that ZFS has quite a bit of (good) overhead to safeguard your data, I then switched to creating a mdadm raid-0 with 18 of the disks (Why not 36? I was really running out of time!)
> At that point I also found 'specrec' from gr-analyze, which seems more suitable. But, even after enlarging its circular buffer to the largest supported values, it would only average a write speed of about 300MB/s.
> In the end I had to settle for recording at only 50MS/s (200MB/s) from only a single channel, a far cry from the 2x 6.4Gb/s I'm ultimately looking to record. Although I did get more than an hour of perfect data out of it, over time the circular buffer did get fuller in bursts, and within 2 hours it exited due to exhausting the buffers. Restarting the application made it work like fresh again, with the same gradual decline
> in performance.
> Specrec, even when tweaking its settings, doesn't really take advantage of the large amount of memory in the server. As a next step, I'm thinking of adapting specrec to use much larger buffers, so that writes are at least in the range of MB to tens of MB. From earlier experiences, it is also important to flush your data to disk often, so the interruptions due to this are more frequent, but short enough to not cause receive buffers to overflow.
> In terms of network tuning, all recording was done with MTU 9000, with wmem and rmem at the recommended values. All recordings were done as interleaved shorts.
> Does anyone have hints or experiences to share?
> Regards, Paul Boven.
> USRP-users mailing list
> USRP-users at lists.ettus.com
More information about the USRP-users