mruane@silveredge-gs.com wrote:

Hello,

Regarding kernel versions (5.15 vs 5.10) I messed that. At the time newest linux-xlnx was 5.15 and USRP X410 used 5.10.

I did my bisecting by starting from xlnx_rebase_v5.10 branch, finding the common commit of NI’s kernel and linux-xlnx (2c85ebc57b3e) and some working commit (top of xlnx_rebase_v5.10 branch). It looked like this:
cd kernel_souces/linux-xlnx/

git bisect reset
git checkout xlnx_rebase_v5.10
git bisect start
git bisect bad
git bisect good 2c85ebc57b3e
git bisect run check_not_working

I’m attaching the scripts used for bisecting in case anyone is interested. They were written ad hoc and not meant to be used by anyone else than me. I.e. ‘git bisect good 2c85ebc57b3e’ means a commit for which ZCU111 failed to boot, so actually opposite of good. But I’m posting them in case anyone finds them helpful. Boot loops were not a problem as the failure to boot was confirmed based on exceeding a timeout.

Regarding the description how to boot X410 over the network - I have written it below, but keep in mind that I didn’t go through all of the steps before writing them down. So most probably I have missed something. When (or if) I’ll have a well tested description, I’ll put it on some webpage and post a link here.

It goes like this (for Ubuntu 20.04):

Install nfs and tftp servers + docker:
sudo apt install nfs-kernel-server tftpd-hpa isc-dhcp-server docker.io

Configure nfs server by putting this in /etc/exports:
/nfsroot *(rw,sync,no_root_squash)
sudo systemctl restart nfs-server

Configure DHCP server. In file /etc/dhcp/dhcpd.conf you need to check if you have something like this for subnet 192.168.0.0:
subnet 192.168.0.0 netmask 255.255.255.0 {
range 192.168.0.20 192.168.0.120;
option subnet-mask 255.255.255.0;
option routers 192.168.0.1;
}

Then restart DHCP server:
sudo systemctl restart isc-dhcp-server

If you need to have constant IP in the USRP add something like to /etc/dhcp/dhcpd.conf:
host x410 {
hardware ethernet 00:80:2f:31:11:0c;
fixed-address 192.168.0.15;
option subnet-mask 255.255.255.0;
option broadcast-address 192.168.0.255;
option routers 192.168.0.1;
}
(replace 00:80:2f:31:11:0c with your USRP’s 1Gb ethernet interface MAC address).

Prepare kernel+bootloaders and rootfs. All of these things are in SD card images provided by NI/Ettus (can be mounted with i.e. kpartx), but I wasn’t able to boot the board over network with use of the kernel provided there. So I’m building whole image myself from meta-ettus repo like this (will take a while):

git clone https://github.com/EttusResearch/meta-ettus
docker run --rm -it -w "$PWD" --mount type=bind,source="$PWD",target="$PWD" \
--net=host \
--env "USER_ID=$(id -u)" \
--env "GROUP_ID=$(id -g)" \
ghcr.io/siemens/kas/kas:2.6.2 \
bash -c "sudo apt update ; sudo apt install -y zip && ./meta-ettus/contrib/kas_build_imgs_package.sh x4xx v4.7.0.0"

Unzip bootloaders to /tftpboot:
sudo unzip build/tmp-glibc/deploy/images/ni-titanium-rev5-mender/u-boot-jtag-files.zip -d /tftpboot

Copy kernel image to /tftpboot:
sudo cp build/tmp-glibc/deploy/images/ni-titanium-rev5-mender/fitImage /tftpboot

Unpack the rootfs to directory exposed by nfs server:
mkdir rootfs_mount_dir
sudo mount build/tmp-glibc/deploy/images/ni-titanium-rev5-mender/gnuradio-image-ni-titanium-rev5-mender-20241005135100.ext4 rootfs_mount_dir -o loop
sudo mkdir /nfsroot/x410
sudo cp -rp rootfs_mount_dir/* /nfsroot/x410

Configure pxelinux to mount rootfs over NFS:
mkdir /tftpboot/x410/pxelinux.cfg
touch /tftpboot/x410/pxelinux.cfg/default

edit file ‘/tftpboot/x410/pxelinux.cfg/default’ and put inside following content (this was taken from Petalinux and edited to mount rootfs from NFS and to have UIO name good for USRP X410):
LABEL Linux
KERNEL fitImage
APPEND earlycon console=ttyPS0,115200 root=/dev/nfs nfsroot=192.168.0.1:/nfsroot/x410,v3,tcp uio_pdrv_genirq.of_id=usrp-uio at24.io_limit=64 rootwait clk_ignore_unused ip=dhcp rw

You need to connect 1Gb Ethernet and JTAG USB port of your X410 to the PC. I assume that the PC’s address on the Ethernet interface connected to X410 is 192.168.0.1 (otherwise change the address in /tftpboot/x410/pxelinux.cfg/default).

Booting:
Rreset the USRP. You can do it over USRP’s microcontroller UART:

stm_uart=$(ls /dev/serial/by-id/usb-Digilent_Digilent_USB_Device_*-if02-port0 | head -n 1)
echo reboot > "$stm_uart"
sleep 0.5
echo zynqmp bootmode jtag > "$stm_uart"
sleep 0.5
echo powerbtn > "$stm_uart"

Then boot the USRP:
cd /tftpboot
source /tools/Xilinx/Vivado/2021.1/settings64.sh
#any other Vivado installation should do, it is only needed for xsdb program that sends bootloaders to ARM’s memory over JTAG
xsdb "boot_u-boot.tcl"

For ZCU111 I did write some u-boot configuration for jtagboot. On X410 the code is not modified to make the netboot easier. You’ll need to connect over UART to the X410:
zynq_uart=$(ls /dev/serial/by-id/usb-Digilent_Digilent_USB_Device_*-if03-port0 | head -n 1)
picocom -b 115200 “$zynq_uart”

So when you’ll see some text written by u-boot type ‘noautoboot‘ to go to u-boot prompt. Then do:
setenv jtagboot 'run netboot'
setenv nfsroot '/nfsroot/x410’
setenv tftproot ''
saveenv

Reset the USRP and boot over JTAG again. Now is a tricky part that I didn’t solve completely for X410 (and for ZCU111 it works well). If everything to this step will go well u-boot will start to look for files in some strange directories in /tftpboot directory. What I did was putting those files there after seeing that u-boot looks for them. At the moment I don’t have a better solution than to show how it looks on my side. So here is content of my /tftpboot (actually I put files for different boards in subdirectories of /tftpboot, but here in order to not complicate this description even more I didn’t add that):
https://ln5.sync.com/dl/1ceadcae0/98ggqg92-7d8mv5cq-txweneu5-pdaqzx26/view/default/14313095050001

Best Regards,
Piotr Krysik

Hi Piotr!

So sorry for the delay!

Wow, that’s a lot of good information!  Thanks for the starting point information.  I spent some time trying to find a common starting point.  I have also been looking at the kernel config parameters in the defconfig and .cfg files.  In the linux-xlnx kernel, there is a CONFIG_XILINX_DPU=y parameter that is be added to a .cfg file in the kernel recipe that magically enables certain DPU-related features.  I knew it couldn’t be as simple as just adding that to the x410 defconfig…but I HAD to try it. :-)  Needless to say, It didn’t work.  I initially tried to go through all of the Kconfig stuff to see if I could figure out what actual kernel parameters CONFIG_XILINX_DPU translated to, but that was a pretty deep rabbit hole that didn’t seem very promising.  So I’m back to comparing the repositories.

Thanks for the scripts.  This truly is a HUGE amount of help.

Regarding network boot:  Now that I’ve seen them again, the tftpboot steps are familiar.  I haven’t done it in a while, and for some reason, I didn’t remember that the bootloader was part of the process.  So it’s the same except for NFS mounting the rootfs.  I’ll start setting that up today.  My build process is slightly different, so I’ll have to adapt some things.

I’m using kas to build, but not in a container.  I just set up a conda environment, and build using the kas commands, like ‘kas build <path-to-custom-x410.yml>’.  Unfortunately, I can’t build the Mender images because there is a bug in either the Mender recipes or the recipe for graphviz_2.50.0.bb that causes a PSEUDO_ABORT error.  It seems that one of the graphviz package directories is deleted outside of PSEUDO, and that causes an inode conflict.

I don’t think it is in the graphviz recipe, even though the pkg_postrm:${PN} task removes the config6 file that is the cause of the conflict.  That task doesn’t seem to run before I get the error.  I have narrowed it down to mender-setup-image.inc, in which the cleanup_excluded_directories function removes the excluded dirs.  For some reason, rootfs.image_tar is getting deleted, and it shouldn’t be.  In the file, mender-part-image.bbclass, all of the Mender-related image tasks are being set to NOT respect the exclude paths, like so:

# So that we can use the files from excluded paths in the full images.
do_image_sdimg[respect_exclude_path] = "0"
do_image_uefiimg[respect_exclude_path] = "0"
do_image_biosimg[respect_exclude_path] = "0"
do_image_gptimg[respect_exclude_path] = "0"

To fix it, I think I just need to let Mender know about the do_image_tar task, and then set that flag to 0.

For now, though, I’m building the non-Mender recipe.

perper@o2.pl wrote:

Hi Piotr,

Thanks.  The PXE boot info is fantastic.  I don’t have it set up yet, but I’m going to start setting everything up today.  I’ll start with just getting JTAG tftpboot running, and then start setting up NFS/PXE boot.

Once this is working, it will REALLY simplify testing the rootfs.  I feel like overwiting the eMMC for an occasional image upgrade is fine, but it’s a bit sketchy to overwrite it several times a day during development work.

:-) That’s why I keep the boards on my desk!  I used to sit at a desk in the lab to be close to the equipment and lab benches, but I moved out because of all of the fan noise in there.  It was deafening.  As soon as I moved out, I missed having all of my boards next to me, so I moved them to my desk…and now my office is almost as loud as the lab.

Cheers,

Mike

Hi Piotr,

I’m really glad you mentioned xlnx_dpu.c and xlnx_dpu.h.  I found those a few days ago (in the linux-xlnx github repo, I think), and I was trying to find a recipe that installs them, but no luck.  I didn’t think of manually copying them to the NI kernel and adding them to the NI recipe.

The  CONFIG_XILINX_DPU parameter is an issue.  It causes a warning that says it’s an unknown object.  I think that’s the parameter that tells “PetaLinux“ to install the DPU drivers, and maybe sets a few other kernel parameters.  My assumption is that I have to figure out what kernel options Xilinx enables when that parameter is set, and set those accordingly.  Kconfig is a mystery to me, so I’ve been putting it off. :-)  I think I need to spend a few hours reading up on Kconfig.  Right now, the Kconfig files just look like menu entries to me.  I don’t see anything that takes those menu entries and writes to defconfig.  Again, I just need to read up on Kconfig while my build is running.

Regarding the patches, I learned how to do that properly during this effort.    I‘ve been writing a lot of them for lately, and I can’t believe it took me so long to try them.  I have to say that kas makes patching REALLY easy.  I used to use ‘repo’ when I had many remote layers or repositories to keep track of.  I’m never using ‘repo’ again.  Kas is great.  Once I fix the image_tar PSEUDO_ABORT issue and can build the Mender images, I’ll set up the Kas docker image, and use the kas_build_imgs_package.sh script.

Thanks for all of the help!  I’ve really learned a lot.

Cheers,

Mike

perper@o2.pl wrote:

Hello Mike,

It’s good that you have experience with tftpboot, because the paths in
my description are a bit wrong. I meant to put everything in /tftpboot
directory for simplicity but then written /tftpboot/x410 - like I have
it. So this is a mistake and ‘/tftpboot/x410’ should be replaced by
‘/tftpboot‘ (for default configuration of the tftp server).

Regarding ‘kas’ command - that works well with the ‘zeus’ branch of
meta-ettus (used up to UHD 4.6). But when I switched to more recent
‘kirkstone’ branch - it stopped. Probably some changes are needed in
that command. I would guess kas verstion 2.6.2 is not correct anymore,
but I don’t know for which the command might work. So it is actually
good that you already are able to run those builds with ‘kas’ but
without docker.

Piotr

W dniu 10.10.2024 o 17:51, mruane--- via USRP-users pisze:

Hi Mike,

The Kconfig files format is probably as mysterious to me (i.e. I don't
know what processes them so they end up in the configuration and has
proper dependencies). But you can just look at what the author of the
driver added to appropriate 'Kconfig' and 'Makefile'. I.e. open 'gitk'
program in linux-xlnx directory and look for commit 5ce32fe84b358a041.
Look for what changes it does to driver/misc/Kconfig and
driver/misc/Makefile.

For experiments it was better for me to compile NI's kernel manually. If
you already performed the build with use of 'kas' then the kernel source
is buried somewhere in your build directory. I have it in:
build/tmp-glibc/work/ni_titanium_rev5_mender-oe-linux/linux-x4xx/5.10.37+gitAUTOINC+e97bd1e03e-r0/linux-ni_titanium_rev5_mender-standard-build/source

One level up is build directory of the kernel. So you can make changes
in the kernel source and then compile it manually.

To make the kernel for zynq you will need to create SDK installer with
use of your yocto build and then install it.
The command that creates the SDK installer is here:
https://github.com/EttusResearch/meta-ettus/blob/kirkstone/contrib/kas_build_imgs_package.sh#L45

But when I try to run that manually (without kas) now, it doesn't work
for some reason (in the directory where you have your yocto build and
'oe-core' directory):
. oe-core/oe-init-build-env
bitbake gnuradio-image -cpopulate_sdk

Then you need to install the SDK (I don't remember in which directory it
will be stored) and source a bash script from the SDK installation dir:
  source oecore-sdk/environment-setup-aarch64-oe-linux

After that all build tools (compilers, etc) will be replaced temporarily
with ARM-compatible ones. So you'll be able to build the kernel with any
manual modifications that you make.

Best Regards,
Piotr Krysik

W dniu 10.10.2024 o 18:41, mruane--- via USRP-users pisze:

On Fri, Oct 11, 2024 at 6:13 AM Piotr Krysik perper@o2.pl wrote:

Can you please open an issue on this on github.com/EttusResearch/uhd? We
use kas to build images, so this should not be an issue.

--M

W dniu 11.10.2024 o 15:19, Martin Braun pisze:

I probably would if I had suspicion that fault is on meta-ettus side. As
I've written I thought that it was my command that got outdated by
recent changes.

After looking into that I figured out that:

• meta-ettus now relies on a much newer version of kas (4.0 ?),
• docker image for new version of kas have changed and don't allow me to
  set the user easily with USER_ID variable to id of current user. Setting
  that was important because it did let me to easily tweak the build
  without changing ownership of the files.

I'm able to use a following hack by making top directory writable by
everyone:

mkdir top_directory
chmod 777 top_directory
cd top_directory
git clone https://github.com/EttusResearch/meta-ettus.git
docker run --rm -it -w "$PWD" --mount
type=bind,source="$PWD",target="$PWD"
           --net=host
           --security-opt seccomp=unconfined
           ghcr.io/siemens/kas/kas:4.0
           bash -c "sudo apt update ; sudo apt install -y zip &&
./meta-ettus/contrib/kas_build_imgs_package.sh x4xx v4.7.0.0"

and the build was successful.

Then I manually built sdk:

docker run --rm -it -w "$PWD" --mount
type=bind,source="$PWD",target="$PWD"
           --net=host
           --security-opt seccomp=unconfined
           ghcr.io/siemens/kas/kas:4.0
           bash

sudo apt update
sudo apt install -y zip
. oe-core/oe-init-build-env
bitbake gnuradio-image -cpopulate_sdk

BTW. this script seems to be responsible for building sdk - in the line 45:
https://github.com/ptrkrysik/meta-ettus/blob/x411/contrib/kas_build_imgs_package.sh#L45

but it's not clear to me in what circumstances this line will be
executed - where _sdk_pkg_name variable will be set?

Best Regards,
Piotr Krysik