[USRP-users] Receiving IO Block Error when Using RFNoC Split Stream

Jonathan Lockhart jlockhartrt at gmail.com
Wed Nov 20 15:24:18 EST 2019


Greetings Nick and EJ,

Yes you are correct, the split stream does require a FIFO after the output
if I want to port to the arm or ZMQ. That resolved the run error.

EJ,

Currently I am just modifying the default installation example that comes
with UHD for the fosphor GNR files, and by default Ettus has it set to 56
MHz, which does appear to be a valid sample rate for the E312 SG3. I
haven't seen any odd behavior with the default example, albeit I am still
playing with the split stream so I may run into issues still.


Regards,
Jon

On Thu, Nov 14, 2019 at 6:11 PM EJ Kreinar <ejkreinar at gmail.com> wrote:

> Cool!
>
> I'll point out that your new build actually got past the old failure point
> you noted before, which was immediately after attempting to interact with
> the SplitStream RFNOC block!
>
> ```
> [INFO] [0/SplitStream_0] Initializing block control (NOC ID:
> 0x5757000000000000)
> [ERROR] [UHD] Exception caught in safe-call.
> [...etc...]
> ```
>
> I second Nick's thought. Try adding a FIFO after the second split-stream
> port.
>
> Though on further inspection, I actually doubt this particular application
> would work at all, since it looks like you're trying to push 56 MHz
> through the RFNoC FPGA->ARM transport and then through ZMQ. The E310
> definitely cannot handle that sort of load, and you might get undefined
> behavior in RFNOC fosphor because the underflow will propagate back to the
> RFNoC Radio and momentarily stop the RF stream before restarting...
>
> EJ
>
> On Thu, Nov 14, 2019 at 6:05 PM Nick Foster <bistromath at gmail.com> wrote:
>
>> I also haven't tested to see if this is a gr-ettus limitation or a UHD
>> limitation.
>>
>> On Thu, Nov 14, 2019 at 3:04 PM Nick Foster <bistromath at gmail.com> wrote:
>>
>>> You can't have heterogenous output destinations on RFNoC blocks -- i.e.,
>>> you can't send one output to the host and one output to another RFNoC block.
>>>
>>> I'm not sure why this limitation exists as there is no architectural
>>> reason I can see.
>>>
>>> Nick
>>>
>>> On Thu, Nov 14, 2019 at 12:35 PM Jonathan Lockhart <
>>> jlockhartrt at gmail.com> wrote:
>>>
>>>> Greetings EJ,
>>>>
>>>> So went and dug out the E312 b/c I couldn't wait and unfortunately that
>>>> didn't resolve the issue I was experiencing. I cherry picked the new
>>>> split_stream block, I am using the same flow graph as provided before, but
>>>> when it goes to run on the E312 I get the following error.
>>>>
>>>> Traceback (most recent call last):
>>>>   File "rfnoc_fosphor_radio_network_usrp.py", line 281, in <module>
>>>>     main()
>>>>   File "rfnoc_fosphor_radio_network_usrp.py", line 271, in main
>>>>     tb.start()
>>>>   File
>>>> "/home/root/localinstall/usr/lib/python2.7/site-packages/gnuradio/gr/top_block.py",
>>>> line 109, in start
>>>>     top_block_start_unlocked(self._impl, max_noutput_items)
>>>>   File
>>>> "/home/root/localinstall/usr/lib/python2.7/site-packages/gnuradio/gr/runtime_swig.py",
>>>> line 3671, in top_block_start_unlocked
>>>>     return _runtime_swig.top_block_start_unlocked(*args, **kwargs)
>>>> RuntimeError: uhd_rfnoc_SplitStream(9): missing connection from output
>>>> port 0
>>>>
>>>> Looks like something is missing from the split stream. I assume it
>>>> needs to be fixed in the verilog.
>>>>
>>>> I attached a screenshot of the new, cleaned up GNU radio file. I had to
>>>> remove a FIFO and switch to using a throttle b/c I wasn't able to get it
>>>> all to fit in the bit file.
>>>>
>>>> Regards,
>>>> Jon
>>>>
>>>> On Wed, Nov 13, 2019 at 10:46 AM Jonathan Lockhart <
>>>> jlockhartrt at gmail.com> wrote:
>>>>
>>>>> Greetings EJ,
>>>>>
>>>>> Thanks for the info. I haven't had time to grab the new block as my
>>>>> dev box is packed for moving this weekend. I will get it downloaded and try
>>>>> it as soon as I can.
>>>>>
>>>>> Regards,
>>>>> Jon
>>>>>
>>>>> On Sat, Nov 9, 2019 at 9:48 AM EJ Kreinar <ejkreinar at gmail.com> wrote:
>>>>>
>>>>>> Hi there,
>>>>>>
>>>>>> Just want to chime in since I recently went through the upgrade
>>>>>> process to UHD-3.14...
>>>>>>
>>>>>> Can you confirm you're using uhd-3.14? If so, this is actually a
>>>>>> split stream fpga bug caused by the commit Jon referred to, not the "not
>>>>>> enough bandwidth" problem.
>>>>>>
>>>>>> Fortunately, rather than referring the old commit, the bug seems to
>>>>>> have been fixed in October on the master branch: https://
>>>>>> github.com/EttusResearch/fpga/commit/1102779f49d44c9e8b88ce7251d203eb62ae26c9
>>>>>> (but not yet ported onto 3.14)
>>>>>>
>>>>>> I just cherry-picked 1102779f onto my UHD-3.14 and it cleaned it up
>>>>>> for me.
>>>>>>
>>>>>> I assume this will eventually make it to the UHD-3.14 branch? But if
>>>>>> not the cherry pick works fine
>>>>>>
>>>>>> EJ
>>>>>>
>>>>>>
>>>>>> On Fri, Nov 8, 2019, 2:43 PM Jonathan Lockhart via USRP-users <
>>>>>> usrp-users at lists.ettus.com> wrote:
>>>>>>
>>>>>>> Jonathon,
>>>>>>>
>>>>>>> I will give that a try and see if it works.
>>>>>>>
>>>>>>>
>>>>>>> Nick,
>>>>>>>
>>>>>>> If the revert on Split_Stream fails, I will try switching from
>>>>>>> ce_clk to bus_clk and give that a try.
>>>>>>>
>>>>>>>
>>>>>>> Regards,
>>>>>>> Jon
>>>>>>>
>>>>>>> On Fri, Nov 8, 2019 at 1:52 PM Nick Foster <bistromath at gmail.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> Jonathon (Pendlum), correct me if I'm wrong, but I think this is
>>>>>>>> the usual split-stream-demands-more-bandwidth-than-RFNoC-bus-allows problem.
>>>>>>>>
>>>>>>>> Jonathan (Lockhart), if I'm right, then in your
>>>>>>>> rfnoc_ce_auto_inst_e310.v, if you change the ce_clk/ce_rst in the
>>>>>>>> noc_block_split_stream instantiation to bus_clk/bus_rst, this may fix the
>>>>>>>> problem.
>>>>>>>>
>>>>>>>> Nick
>>>>>>>>
>>>>>>>> On Fri, Nov 8, 2019 at 10:20 AM Jonathon Pendlum <
>>>>>>>> jonathon.pendlum at ettus.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Jon,
>>>>>>>>>
>>>>>>>>> Can you try reverting commit e39334fe on the fpga repo and
>>>>>>>>> rebuilding your bitstream? Let me know if that makes a difference or not.
>>>>>>>>>
>>>>>>>>> Jonathon
>>>>>>>>>
>>>>>>>>> On Sat, Nov 9, 2019 at 12:13 AM Jonathan Lockhart via USRP-users <
>>>>>>>>> usrp-users at lists.ettus.com> wrote:
>>>>>>>>>
>>>>>>>>>> Greetings Nick,
>>>>>>>>>>
>>>>>>>>>> Here is the screenshot of my GR flow graph, though it shouldn't
>>>>>>>>>> matter as the Split_Stream RFNoC Block I believe is a failure of the python
>>>>>>>>>> or Verilog but the solutions in the link I sent in my previous email did
>>>>>>>>>> not resolve the issue.
>>>>>>>>>>
>>>>>>>>>> [image: Screenshot from 2019-11-08 10-06-50.png]
>>>>>>>>>>
>>>>>>>>>> Regards,
>>>>>>>>>> Jon
>>>>>>>>>>
>>>>>>>>>> On Thu, Nov 7, 2019 at 5:33 PM Nick Foster <bistromath at gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> Can you be more specific about what your flowgraph looks like?
>>>>>>>>>>>
>>>>>>>>>>> On Thu, Nov 7, 2019 at 2:22 PM Jonathan Lockhart via USRP-users <
>>>>>>>>>>> usrp-users at lists.ettus.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Greetings,
>>>>>>>>>>>>
>>>>>>>>>>>> I was wondering if anyone had encountered the following error
>>>>>>>>>>>> and had a way to fix it?
>>>>>>>>>>>>
>>>>>>>>>>>> [INFO] [UHD] linux; GNU C++ version 4.9.2; Boost_105700;
>>>>>>>>>>>> UHD_3.14.1.HEAD-0-g0347a6d8
>>>>>>>>>>>> [INFO] [E300] Loading FPGA image:
>>>>>>>>>>>> /home/root/localinstall/e300.bit...
>>>>>>>>>>>> [INFO] [E300] FPGA image loaded
>>>>>>>>>>>> [INFO] [E300] Detecting internal GPS
>>>>>>>>>>>> .... [INFO] [E300] GPSDO found
>>>>>>>>>>>> [INFO] [E300] Initializing core control (global registers)...
>>>>>>>>>>>>
>>>>>>>>>>>> [INFO] [E300] Performing register loopback test...
>>>>>>>>>>>> [INFO] [E300] Register loopback test passed
>>>>>>>>>>>> [INFO] [0/Radio_0] Initializing block control (NOC ID:
>>>>>>>>>>>> 0x12AD100000000000)
>>>>>>>>>>>> [WARNING] [RFNOC] Can't find a block controller for key
>>>>>>>>>>>> SplitStream, using default block controller!
>>>>>>>>>>>> [INFO] [0/SplitStream_0] Initializing block control (NOC ID:
>>>>>>>>>>>> 0x5757000000000000)
>>>>>>>>>>>> [ERROR] [UHD] Exception caught in safe-call.
>>>>>>>>>>>>   in ctrl_iface_impl<_endianness>::~ctrl_iface_impl() [with
>>>>>>>>>>>> uhd::endianness_t _endianness = (uhd::endianness_t)1u]
>>>>>>>>>>>>   at
>>>>>>>>>>>> /home/jon/rfnoc_3.14.1.1/src/uhd/host/lib/rfnoc/ctrl_iface.cpp:52
>>>>>>>>>>>> this->send_cmd_pkt(0, 0, true); -> EnvironmentError: IOError:
>>>>>>>>>>>> Block ctrl (CE_01_Port_21) no response packet - AssertionError: bool(buff)
>>>>>>>>>>>>   in uint64_t ctrl_iface_impl<_endianness>::wait_for_ack(bool,
>>>>>>>>>>>> double) [with uhd::endianness_t _endianness = (uhd::endianness_t)1u;
>>>>>>>>>>>> uint64_t = long long unsigned int]
>>>>>>>>>>>>   at
>>>>>>>>>>>> /home/jon/rfnoc_3.14.1.1/src/uhd/host/lib/rfnoc/ctrl_iface.cpp:142
>>>>>>>>>>>>
>>>>>>>>>>>> Traceback (most recent call last):
>>>>>>>>>>>>   File "rfnoc_fosphor_network_usrp.py", line 283, in <module>
>>>>>>>>>>>>     main()
>>>>>>>>>>>>   File "rfnoc_fosphor_network_usrp.py", line 272, in main
>>>>>>>>>>>>     tb = top_block_cls(fft_size=options.fft_size,
>>>>>>>>>>>> fpga_path=options.fpga_path, freq=options.freq, gain=options.gain,
>>>>>>>>>>>> host_ip_addr=options.host_ip_addr, samp_rate=options.samp_rate,
>>>>>>>>>>>> tdecay=options.tdecay, trise=options.trise)
>>>>>>>>>>>>   File "rfnoc_fosphor_network_usrp.py", line 43, in __init__
>>>>>>>>>>>>     self.device3 = variable_uhd_device3_0 =
>>>>>>>>>>>> ettus.device3(uhd.device_addr_t( ",".join(('type=e3x0',
>>>>>>>>>>>> "master_clock_rate=%d,fpga=%s" % (samp_rate,fpga_path))) ))
>>>>>>>>>>>>   File
>>>>>>>>>>>> "/home/root/localinstall/usr/lib/python2.7/site-packages/ettus/ettus_swig.py",
>>>>>>>>>>>> line 1307, in make
>>>>>>>>>>>>     return _ettus_swig.device3_make(*args, **kwargs)
>>>>>>>>>>>> RuntimeError: EnvironmentError: IOError: [0/SplitStream_0]
>>>>>>>>>>>> sr_read64() failed: EnvironmentError: IOError: Block ctrl (CE_01_Port_21)
>>>>>>>>>>>> no response packet - AssertionError: bool(buff)
>>>>>>>>>>>>   in uint64_t ctrl_iface_impl<_endianness>::wait_for_ack(bool,
>>>>>>>>>>>> double) [with uhd::endianness_t _endianness = (uhd::endianness_t)1u;
>>>>>>>>>>>> uint64_t = long long unsigned int]
>>>>>>>>>>>>   at
>>>>>>>>>>>> /home/jon/rfnoc_3.14.1.1/src/uhd/host/lib/rfnoc/ctrl_iface.cpp:142
>>>>>>>>>>>>
>>>>>>>>>>>> This is only occurring when I use the split stream block in
>>>>>>>>>>>> RFNoC. I have tried the fixes in [1] but unfortunately they have not
>>>>>>>>>>>> helped. Also, fix 1, I can't seem to before b/c I am missing the file
>>>>>>>>>>>> rfnoc_ce_auto_inst_<device>.v  as shown from the output when
>>>>>>>>>>>> attempting a "find" command in Ubuntu.
>>>>>>>>>>>>
>>>>>>>>>>>> find: ‘rfnoc_ce_auto_inst_e310.v’: No such file or directory
>>>>>>>>>>>>
>>>>>>>>>>>> I ran it on both ~/* and /* with no luck.
>>>>>>>>>>>>
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Jon Lockhart
>>>>>>>>>>>>
>>>>>>>>>>>> [1]
>>>>>>>>>>>> https://kb.ettus.com/RFNoC#Why_do_I_have_a_command_timeout_error.3F
>>>>>>>>>>>>
>>>>>>>>>>>> _______________________________________________
>>>>>>>>>>>> USRP-users mailing list
>>>>>>>>>>>> USRP-users at lists.ettus.com
>>>>>>>>>>>>
>>>>>>>>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
>>>>>>>>>>>>
>>>>>>>>>>> _______________________________________________
>>>>>>>>>> USRP-users mailing list
>>>>>>>>>> USRP-users at lists.ettus.com
>>>>>>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>> USRP-users mailing list
>>>>>>> USRP-users at lists.ettus.com
>>>>>>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
>>>>>>>
>>>>>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/attachments/20191120/5a1ebbe0/attachment.html>


More information about the USRP-users mailing list