Discussion and technical support related to USRP, UHD, RFNoC
View all threadsHi Piotr,
I’m using kas 4.4. I followed the README in github.com/meta-ettus, under the section, “Building an Image using the Alchemy distro and kas.“ There is a link to the kas repo, and I just installed the latest.
It’s been a while, but I believe that I went the manual kas route because I had a hard time with the kas container. I know that was true when I originally tried the zeus branch.
Cheers,
Mike
Piotr Krysik wrote:
Hello Mike,
It’s good that you have experience with tftpboot, because the paths in
my description are a bit wrong. I meant to put everything in /tftpboot
directory for simplicity but then written /tftpboot/x410 - like I have
it. So this is a mistake and ‘/tftpboot/x410’ should be replaced by
‘/tftpboot‘ (for default configuration of the tftp server).
Regarding ‘kas’ command - that works well with the ‘zeus’ branch of
meta-ettus (used up to UHD 4.6). But when I switched to more recent
‘kirkstone’ branch - it stopped. Probably some changes are needed in
that command. I would guess kas verstion 2.6.2 is not correct anymore,
but I don’t know for which the command might work. So it is actually
good that you already are able to run those builds with ‘kas’ but
without docker.
Piotr
W dniu 10.10.2024 o 17:51, mruane--- via USRP-users pisze:
Hi Piotr!
So sorry for the delay!
Wow, that’s a lot of good information! Thanks for the starting point
information. I spent some time trying to find a common starting point.
I have also been looking at the kernel config parameters in the
defconfig and .cfg files. In the linux-xlnx kernel, there is a
CONFIG_XILINX_DPU=y parameter that is be added to a .cfg file in the
kernel recipe that magically enables certain DPU-related features. I
knew it couldn’t be as simple as just adding that to the x410
defconfig…but I HAD to try it. :-) Needless to say, It didn’t work. I
initially tried to go through all of the Kconfig stuff to see if I
could figure out what actual kernel parameters CONFIG_XILINX_DPU
translated to, but that was a pretty deep rabbit hole that didn’t seem
very promising. So I’m back to comparing the repositories.
Thanks for the scripts. This truly is a HUGE amount of help.
Regarding network boot: Now that I’ve seen them again, the tftpboot
steps are familiar. I haven’t done it in a while, and for some reason,
I didn’t remember that the bootloader was part of the process. So it’s
the same except for NFS mounting the rootfs. I’ll start setting that
up today. My build process is slightly different, so I’ll have to
adapt some things.
I’m using kas to build, but not in a container. I just set up a conda
environment, and build using the kas commands, like ‘kas build
<path-to-custom-x410.yml>’. Unfortunately, I can’t build the Mender
images because there is a bug in either the Mender recipes or the
recipe for graphviz_2.50.0.bb that causes a PSEUDO_ABORT error. It
seems that one of the graphviz package directories is deleted outside
of PSEUDO, and that causes an inode conflict.
I don’t think it is in the graphviz recipe, even though the
pkg_postrm:${PN} task removes the config6 file that is the cause of
the conflict. That task doesn’t seem to run before I get the error. I
have narrowed it down to mender-setup-image.inc, in which the
cleanup_excluded_directories function removes the excluded dirs. For
some reason, rootfs.image_tar is getting deleted, and it shouldn’t be.
In the file, mender-part-image.bbclass, all of the Mender-related
image tasks are being set to NOT respect the exclude paths, like so:
|# So that we can use the files from excluded paths in the full
images. do_image_sdimg[respect_exclude_path] = "0"
do_image_uefiimg[respect_exclude_path] = "0"
do_image_biosimg[respect_exclude_path] = "0"
do_image_gptimg[respect_exclude_path] = "0"|
To fix it, I think I just need to let Mender know about the
do_image_tar task, and then set that flag to 0.
For now, though, I’m building the non-Mender recipe.
Hey Piotr!
I was able to patch the meta-mender layer to fix the PSEUDO_ABORT bug. I’m still not sure it’s a bug in Mender, but it only seems to occur when I have the graphviz recipe included, which is required by the Vitis-AI library recipes. I’m really glad to have Mender images now. That will simplify some things for me.
I’ll make a fork of the meta-ettus and other repos on github, and start pushing commits up to it.
Cheers
Mike
Hi Piotr,
I’m looking into the DPU driver kernel mods now. I’ll keep you posted.
Cheers,
Mike
perper@o2.pl wrote:
Hello Mike,
After short look at current linux-xlnx - it seems that there are not that many changes needed for DPU to work. The driver is in one C file + C header:
linux-xlnx/drivers/misc/xlnx_dpu.c
linux-xlnx/drivers/misc/xlnx_dpu.h
Look at their histories. Initial addition of this driver is in commit 5ce32fe84b358a041. Previous commit adds device-tree documentation. If I were doing it myself I would probably first manually copy those files to NI’s kernel + associated changes in driver/misc/Kconfig and driver/misc/Makefile
then add CONFIG_XILINX_DPU=y option in the configuration file and compile the kernel manually. This would probably verify if those two files are all what is required for DPU to work. Only after that I would add the commits from linux-xlnx to NI’s kernel source (by cherry-picking them). After that I would turn them into a series of patches that could be copied into meta-titanium kernel.
Best Regards,
Piotr Krysik
Hi Piotr, Martin,
I’ve had some success in getting the driver installed in the kernel. It is pretty straightforward, and Piotr’s suggestions regarding xlnx_dpu.c
and xlnx_dpu.h
were spot on.
Currently, the driver fails while loading during boot. Some Google searches suggest either the wrong driver version, or a device tree issue. I’m pretty sure that I’m compiling the correct driver version. I iterated through several versions, from early to more recent, and I am currently building the v2022.2 version, which matches the DPU version in my design, which is 4.1.
I suspect that there may be an issue with my device tree, as there is little information regarding the proper DPU device tree entries. My guess is that I’ve messed up an interrupt, or stomped on an address that the original x410 RFDC block design is using. I tried to keep all of the existing addresses the same, and only add a new, unused address for the DPU, but I’ve had Vivado IP integrator change the addresses a couple of times after I’ve modified the BD, so I’m checking everything to be sure.
If you are interested, I pushed the meta_ettus
changes to my fork at https://github.com/tetonicus/meta-ettus. I didn’t separate the commits in the fork, I just pushed them as “Initial Commit.” The kernel-driver-related changes are primarily in meta-ettus-bsp/recipes-kernel/linux/linux-x4xx/
.
I’ll post more when I get the driver to load…or…if something goes horribly wrong and I think it would be useful to share. :-)
Cheers,
Mike
Hello Mike,
When I did the port the most cumbersome part of it was device-tree (i.e. in terms of troubles-per-character). Probably the one reason was that I didn’t know all the conventions that are used there. Or maybe that some conventions were a bit illusory and each driver had its own. So when you say it might be because of device-tree my immediate reaction is ‘it’s ALWAYS device-tree’s fault‘.
With the interrupts - the convention in Linux was silly as there is some offset between ZYNQ interrupt numbers and Linux’s interrupt numbers. I don’t remember what it was exactly, but something above 80. You can check what interrupt numbers are used with: cat /proc/interrupts
Regarding the network boot - I remember now why I didn’t use the netboot_pxe - because in comparison with NI’s netboot it didn’t set some important variables like revision number. As a result wrong device-tree was loaded - one of the symptoms can be seen on the control UART port. The builtin micro-controller prints a lot of errors.
So that more work is required to make PXE boot working and NI’s netboot should be used for now. I don’t yet know how to make it more clean and robust. Currently it seems a bit messy, but at least it works.
Best Regards,
Piotr Krysik
Hi Piotr!
That was good timing. I was just getting ready to post an update.
my immediate reaction is ‘it’s ALWAYS device-tree’s fault‘.
Hahahaha You are SO right! I have the same issues of familiarity with the device tree conventions as you do. Your ‘troubles-per-character` unit of measurement is hysterical, and perfect for measuring device-tree frustrations. :-) In my case, it was indeed the interrupts. After checking that I didn’t mess up the address map, I started looking at the interrupts. As it turns out, the interrupts that were assigned in the Hardware Definition File (XSA), presumably by the Vivado IP Integrator in the Block Design, were incorrect.
After some time with my nose in the MPSoC Technical Reference Manual (UG1085), I learned that the two PL-to-PS interrupt ports have a fixed bit-to-interrupt assignment. The x410 FPGA design is using almost every bit from both interrupt ports, and they arrive at the interrupt ports through multiple levels of concatenation blocks. I’m not sure, but it it appears as though the IP Integrator tool didn’t correctly trace the individual interrupt paths from their sources to the port pins. The tool had assigned interrupts 104 and 105 to the DPU and SoftMax cores, but these were already connected to the nixge_internal: ethernet@10000A4000
node. The pins that the DPU/SFM core interrupts traced back to were assigned to interrupts 110 and 111. Once I updated the DTS file in the FPGA build (which I used a recipe to tell Yocto to pull into the immage), the driver successfully loaded at boot. Here is the interrupt map from UG1085:
Best,
Mike
Hello Mike,
One thing that sped up development for me regarding device-tree was ability to reload it quickly after changes. The quickest option to load device-tree for FPGA code is to modify related device-tree overlay, compile it manually and reload it without restarting the device. To compile default device-tree you can execute following commands from ‘uhd/fpga/usrp3/top/x400/dts’ directory:
gcc -I ./ -E -nostdinc -undef -D__DTS__ -x assembler-with-cpp -o usrp_x410_fpga_X4.dts.preprocessed usrp_x410_fpga_X4.dts
dtc -I dts -O dtb -o x410_X4.dtbo -b 0 -@ usrp_x410_fpga_X4.dts.preprocessed
Your modifications you can put in some file and include it in ‘usrp_x410_fpga_X4.dts’.
The ‘x410_X4.dtbo’ file can be copied to ‘/lib/firmware’ directory in USRP’s filesystem. You can back-up your ‘x410.dtbo’ and create symbolic link to ‘x410.dtbo -> x410_X4.dtbo’.
The overlay is reloaded when ‘usrp-hwd’ service (which executes ‘usrp_hwd.py’ program) is loaded:
systemctl restart usrp-hwd
The DT overlay can be reloaded manually in bash. I’m using following script for this:
OVERLAY_NAME=”x410_X4"
echo ${OVERLAY_NAME}
SYSFS_OVERLAY_BASE_DIR=/sys/kernel/config/device-tree/overlays
if [ -d ${SYSFS_OVERLAY_BASE_DIR}/${OVERLAY_NAME} ]; then
rmdir ${SYSFS_OVERLAY_BASE_DIR}/${OVERLAY_NAME}
fi
rmdir ${SYSFS_OVERLAY_BASE_DIR}/*
cp -f ${OVERLAY_NAME}.{bin,dtbo} /lib/firmware
mkdir ${SYSFS_OVERLAY_BASE_DIR}/${OVERLAY_NAME}
echo ${OVERLAY_NAME}.dtbo > ${SYSFS_OVERLAY_BASE_DIR}/${OVERLAY_NAME}/path
Another useful script is ‘fpga_bit_to_bin.py’ that is somewhere in UHD. It is much faster and convenient to use than Vivado’s tcl scripts for conversion between ‘bit’ format (that Vivado generates) and ‘bin’ format (that is expected by ‘usrp_hwd.py’).
Best Regards,
Piotr Krysik