usrp-users@lists.ettus.com
Discussion and technical support related to USRP, UHD, RFNoC
View all threadsx410 stuck in reboot
Hello,
I recently tried to update the x410 FPGA over SSH. However, I was met with a strange error, and now when I try to log into the x410, even over console JTAG, I am unable to. It appears the x410 is caught in a reboot loop.
Below is the message I got when trying to update the x410.
[ERROR] [UHD] An unexpected exception was caught in a task loop.The task loop will now exit, things may not work.rpc::timeout: Timeout of 10000ms while calling RPC function 'reclaim'
[ERROR] [UHD] Exception caught in safe-call.
in ~mpmd_mboard_impl
at /home/workarea/uhd/host/lib/usrp/mpmd/mpmd_mboard_impl.cpp:322
dump_logs(); _claimer_task.reset(); if (not rpc->request_with_token<bool>("unclaim")) { uhd::_log::log(uhd::log::warning, "/home/frosty/workarea/uhd/host/lib/usrp/mpmd/mpmd_mboard_impl.cpp", 324, "MPMD", std::this_thread::get_id()) << "Failure to ack unclaim!";; } -> rpc::timeout: Timeout of 10000ms while calling RPC function 'get_log_buf'
Error: rpc::timeout: Timeout of 120000ms while calling RPC function 'update_component'
Thanks
Joe
After reflashing the eMMC, boot is no longer on loop, and I am able to log in via Console JTAG. However, I cant seem to ssh into the device of ethernet or SFP.
I get the following error in boot…
[FAILED] Failed to start File System Check on Root Device.
See 'systemctl status systemd-fsck-root.service' for details.
I am currently trying to flash the x410 with the default x410 FPGA image, but the host computer can not detect uhd devices currently.
I was eventually able to resolve the “[FAILED] Failed to start File System Check on Root Device.” using fsck tools.
I still am not able to ssh into the x410 however. The addresses to not automatically change to the default values(i.e sfp0 = 192.168.10.2), and manually changing them does not appear to work either.
On 14/11/2023 17:40, jmaloyan@umass.edu wrote:
I was eventually able to resolve the “[FAILED] Failed to start File
System Check on Root Device.” using fsck tools.
I still am not able to ssh into the x410 however. The addresses to not
automatically change to the default values(i.e sfp0 = 192.168.10.2),
and manually changing them does not appear to work either.
USRP-users mailing list -- usrp-users@lists.ettus.com
To unsubscribe send an email to usrp-users-leave@lists.ettus.com
The management console of the X410 (if it's like others of its type)
uses DHCP by default, and you'll have to figure out what
address to use based on that. If it's like other similar devices (I
don't have one in my collection yet), you cannot
SSH in via the SFP ports--they are strictly for data streaming, and
certainly not for mundane tasks like SSH management.
I've just tried SSH'ing into an X410 via the SFP ports and that worked. Nevertheless, I'd prefer using the 1 Gbps interface.
You wrote you are unable to use the JTAG port to connect to the device. What does that mean? Can you connect to the serial port exposed by this, but you cannot login to the embedded Linux or are you completely unable to connect? If you cannot connect to the serial console then removing the power supply for some time, reconnecting and switching the X410 on again might help, too.
I don't know what is going on with this device, but if it's really a boot loop and you can at least connect to the JTAG while booting up you could try this:
https://files.ettus.com/manual/page_usrp_x4xx.html#x4xx_resetting_boot_environment
If that doesn't help but you have the JTAG access, you should try re-imaging the device:
https://files.ettus.com/manual/page_usrp_x4xx.html#x4xx_accessing_emmc_usb
Unless you have a lot of work on the device that isn't saved anywhere else, I wouldn't spend any time on restoring the file system but rather flash the device again.
As a last resort if you cannot get the device back up and running, please reach out to our support team.
/Martin
-----Original Message-----
From: Marcus D. Leech patchvonbraun@gmail.com
Sent: Wednesday, November 15, 2023 12:15 AM
To: usrp-users@lists.ettus.com
Subject: [USRP-users] Re: x410 stuck in reboot
On 14/11/2023 17:40, jmaloyan@umass.edu wrote:
I was eventually able to resolve the “[FAILED] Failed to start File
System Check on Root Device.” using fsck tools.
I still am not able to ssh into the x410 however. The addresses to not
automatically change to the default values(i.e sfp0 = 192.168.10.2),
and manually changing them does not appear to work either.
USRP-users mailing list -- usrp-users@lists.ettus.com To unsubscribe
send an email to usrp-users-leave@lists.ettus.com
The management console of the X410 (if it's like others of its type) uses DHCP by default, and you'll have to figure out what
address to use based on that. If it's like other similar devices (I don't have one in my collection yet), you cannot
SSH in via the SFP ports--they are strictly for data streaming, and certainly not for mundane tasks like SSH management.
USRP-users mailing list -- usrp-users@lists.ettus.com To unsubscribe send an email to usrp-users-leave@lists.ettus.com
National Instruments Dresden GmbH; Geschäftsführer (Managing Directors): John Stanton McElroy, Albert Edward Percival III, Kathleen Spurck; Sitz (Registered Office): Dresden; HRB (Commercial Register No.): 22081; Registergericht (Registration Court): Dresden
This email and any attachments are intended only for the person to whom this email is addressed and may contain confidential and/or privileged information. If you received this email in error, please do not disclose the contents to anyone, but notify the sender by return email and delete this email (and any attachments) from your system.
Previously I was unable to log into linux due to the boot loop. However, after reflashing the eMMC with the filesystem(https://kb.ettus.com/USRP_X410/X440_Getting_Started_Guide#Flashing_the_eMMC) I am able to now log into linux, but only through Console JTAG
I can NOT however, log in via SFP or the 1Gb ethernet. This whole problem started after I tried to upload an FPGA bitstream to the ettus, is it possible the x410 relies on the FPGA bitstream to communicate over SFP/1 Gb ethernet at all?
On 15/11/2023 09:40, jmaloyan@umass.edu wrote:
Previously I was unable to log into linux due to the boot loop.
However, after reflashing the eMMC with the
filesystem(https://kb.ettus.com/USRP_X410/X440_Getting_Started_Guide#Flashing_the_eMMC)
I am able to now log into linux, but only through Console JTAG
I can NOT however, log in via SFP or the 1Gb ethernet. This whole
problem started after I tried to upload an FPGA bitstream to the
ettus, is it possible the x410 relies on the FPGA bitstream to
communicate over SFP/1 Gb ethernet at all?
If the implementation is like other, similar, radios, then, yes. The SFP
10G implementation MAC is done in the FPGA code, and if
that FPGA code is not working, then that would cause the SFP comms to
not work. I'm not sure on the X410 whether the
1G RJ-45 management port is "hardware inherent" or not.
USRP-users mailing list -- usrp-users@lists.ettus.com
To unsubscribe send an email to usrp-users-leave@lists.ettus.com
I do have access to two x410s. The bitstream I tried to upload on this currently none working device was CG bitstream, when prior to this it was an X4 bitstream. Im obviously hesitant to try something similar on the working device right now, but my current guess right now is that ssh is not starting.
For example, if I use systemctl -l --type service --all|grep ssh , i get the following for the non-working and working x410s
Non-working:
* sshd.service not-found inactive dead sshd.service
sshd@0.service loaded inactive dead OpenSSH Per-Connection Daemon
sshdgenkeys.service loaded active exited OpenSSH Key Generation
Working :
● sshd.service not-found inactive dead sshd.service
sshd@3-192.168.10.2:22-192.168.10.1:45278.service loaded active running OpenSSH Per-Connection Daemon (192.168.10.1:45278)
sshdgenkeys.service loaded active exited OpenSSH Key Generation
It appears the sshd session was able to start on the working device, but for whatever reason-even after re-flashing- the non-working device is unable to start ssh, though I am not sure how to manually start it.
I have tried using systemctl to start it, but I just get met with an error that the sshd service is not found
On 15/11/2023 13:54, jmaloyan@umass.edu wrote:
I do have access to two x410s. The bitstream I tried to upload on this
currently none working device was CG bitstream, when prior to this it
was an X4 bitstream. Im obviously hesitant to try something similar on
the working device right now, but my current guess right now is that
ssh is not starting.
Ah, this is critical information.
My guess is that whatever filesystem image you re-flashed is broken in
some way so that sshd.service isn't there.
Presumably, you can ping the device? I was previously left with the
impression that there was NO network connectivity
to the device.
For example, if I use |systemctl -l --type service --all|grep ssh| , i
get the following for the non-working and working x410s
Non-working:
- sshd.service not-found inactive dead sshd.service
sshd@0.service loaded inactive dead OpenSSH Per-Connection Daemon
sshdgenkeys.service loaded active exited OpenSSH Key Generation
Working :
● sshd.service not-found inactive dead sshd.service
sshd@3-192.168.10.2:22-192.168.10.1:45278.service loaded active
running OpenSSH Per-Connection Daemon (192.168.10.1:45278)
sshdgenkeys.service loaded active exited OpenSSH Key Generation
It appears the sshd session was able to start on the working device,
but for whatever reason-even after re-flashing- the non-working device
is unable to start ssh, though I am not sure how to manually start it.
I have tried using |systemctl| to start it, but I just get met with an
error that the sshd service is not found
USRP-users mailing list --usrp-users@lists.ettus.com
To unsubscribe send an email tousrp-users-leave@lists.ettus.com
I can not ping the non-working device even, either through RJ-45 or the SFP port. To be clear, everything I am doing to network into the non-working device is through console JTAG. Before this problem started, I was able to ssh into the processing system(Embedded linux) through either RJ-45 port and the SFP ports just fine.
In general, the x410 automatically sets the SFP ports 1-4 to 192.168.10.1-4 respectively. This is something I generally rely to not only log into the device, but when I want to run UHD software. If I ever want to use the RJ-45 port, I usually have to manually set the ip address using ifconfig, but this is pretty trivial.
Now, the device does not set the SFP addresses automatically, and I suspect it has something to do with the ssh service not starting. Additionally, if I use the Console JTAG port to manually set the address of the RJ-45 port, then try to ping that address from my host device, the address is not reachable.