[USRP-users] Synchronised Recording across different systems (OpenMPI)

Carel Combrink carel.combrink at gmail.com
Thu Dec 11 04:45:53 EST 2014

Hi Matt,

Working on your suggestion I found the problem. It was not a problem with
UHD but with OpenMPI. Everything started correctly but after 5 seconds of
capturing my receiver and recorder threads started to catch up with each
other and things started to break down.

For completeness the solution, though not related to UHD:
Further investigation revealed that OpenMPI caused the app to be confined
to 1 CPU core on the TK1 thus all threading were serialised. I had to
configure OpenMPI to use all cores on the TK1 and everything seems fine now
(using --cpus-per-proc command line argument).


On Thu, Dec 11, 2014 at 9:05 AM, Matt Ettus <matt at ettus.com> wrote:

> Carel,
> I don't have any experience with OpenMPI, but it sounds like that version
> is slower and is thus sending the commands late.  Does it work if you don't
> use timed commands?
> Matt
> On Wed, Dec 10, 2014 at 8:15 AM, Carel Combrink via USRP-users <
> usrp-users at lists.ettus.com> wrote:
>> Hi,
>> I am trying to synchronise recording over different systems from
>> different B210 devices. I have connected and set up the B210 devices to
>> different NVIDIA Jetson TK1 boards.
>> So to synchronise the recording I am trying to use OpenMPI, which I have
>> set up and is working in principle.
>> When I run the application distributed using OpenMPI I get the following
>> error when starting to stream:
>> My code flows as follow (extracts of relevant parts):
>> -------------------
>> usrp1 = uhd::usrp::multi_usrp::make(deviceArgs);
>> usrp1->set_clock_source(std::string("internal"));
>> /* ... more setup like freq and gain */
>> usrp1->set_time_now(uhd::time_spec_t(0.0));
>> hd::rx_streamer::sptr rxStream = usrp1->get_rx_stream(streamArgs);
>> -------------------
>> and then in a different thread:
>> -------------------
>> uhd::stream_cmd_t
>> streamCommand(uhd::stream_cmd_t::STREAM_MODE_START_CONTINUOUS);
>> streamCommand.stream_now = false;
>> streamCommand.time_spec = uhd::time_spec_t(10);
>> rxStream->issue_stream_cmd(streamCommand);
>> rxStream->recv(buffPtrs, buffers.numberSamples, md, 10.1);
>> --------------------
>> If I print out the time of the metadata it is: 15.0048. This is almost
>> always 5 seconds after the stream command that I set up.
>> After the initial error I get ERROR_CODE_TIMEOUT without ever receiving
>> data.
>> As mentioned I am using OpenMPI, but when I run the same code on my
>> device (TK1, not through OpenMPI but same executable) it goes through fine
>> and I record perfectly.
>> Are there anything that I am missing that would cause this to happen?
>> PS: I am only using basic OpenMPI to start the apps remotely and to
>> synchronise at some points, like before issuing the stream command. There
>> are no synchronising between the apps after issuing the stream command. I
>> have even removed all OpenMPI code from the application but the same
>> problem exists.
>> Regards,
>> _______________________________________________
>> USRP-users mailing list
>> USRP-users at lists.ettus.com
>> http://lists.ettus.com/mailman/listinfo/usrp-users_lists.ettus.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ettus.com/pipermail/usrp-users_lists.ettus.com/attachments/20141211/02e71597/attachment-0002.html>

More information about the USRP-users mailing list