Yeah…I’ll have to look at that.  I’m actually building UHD 4.7.0, but there’s nothing newer than 5.10 in meta-ettus, and my current 4.7 image on the board is showing 5.10.37-x4xx.  I’m building the FPGA design with Vivado 2022.2, so I had to upgrade all of the IP, but it all went as smoothly as any other tool upgrade. :-)

I just cloned the linux-xlnx repo, and I’m looking through it now.  I’ll check the kernel version that they’re building in the 2022.2 tools and start there.

I’m going to start poking at the repo with ‘git bisect‘, so if you hear any grumbling, it’s probably me, continually cloning over the mistakes I made. :-)

Mike

Hi Piotr,

I have ZERO experience modifying the bootloader.  I’ve used TFTPBoot before, for bootloader, kernel, and rootfs, and it worked well.  I’ve never mounted a remote rootfs over NFS, or loaded the bootloader over JTAG.  I saw some places in the code where you made some NFS- and JTAG-related changes.  I was thinking I would not use that stuff because I have a board on my desk, but…mounting the eMMC over USB, unzipping the ext4.tar.gz, and overwriting the correct partition on the eMMC is a REAL pain.  And it’s only a matter of time before I brick the device because I was rushing through the steps.

Ok…THAT is a great idea.  I’ve been avoiding ‘git bisect’ all this time.  Hahaha  I’ll feel guilty for asking how you set that up, but I think I will be too curious not to.

You know, with a little scripting, it might be possible to remotely monitor something on the board that gives an indication of whether it successfully booted, like RESET BOARD, WAIT FOR N SECONDS, PING BOARD…, and then automatically tell ‘git bisect’ to keep going or not.  If you were only looking for a successful boot, and you weren’t worried about what the exact failures were, you could just launch it and work on something else.  Hahaha  Sometimes being lazy can be a lot of work for me.

Mike

Hey all,

thanks for the feedback on our build system. Mike, I'm glad to hear you
weren't put off by the rfnoc_image_builder workflow on top of make (I
assume you've been using that given the UHD 4.7 dependency?) and it's nice
to hear you've been successful building stuff.

I had actually checked out hdlmake as a potential amendment for our build
system in the past, and it didn't really fit. For example, re-targeting
FPGAs (we need this, e.g., for the X300 vs. X310 builds, but also for the
X410 vs. X440 builds where we share IP between devices) wasn't
straightforward. Despite all the annoying things that come with CMake (who
came up with that syntax?) it has brought a bunch of niceties to software
builds that I think the FPGA build world is still missing. We had also
looked at scons in the past, but although we're generally Python fans it
didn't jibe with us.

That said if you find something unnecessarily annoying with our build
system, please let us know. We know that building custom bitfiles for USRPs
hasn't always been the easiest/nicest option in the past, and we would
really like to make the FPGA computational capacities available to more
people.

--M

On Thu, Oct 3, 2024 at 7:23 PM mruane--- via USRP-users <
usrp-users@lists.ettus.com> wrote:

Be advised that bad configurations of kernel and/or FPGA (or device tree
info) can lead to boot loops which are pretty annoying to auto-fix.

--M

On Thu, Oct 3, 2024 at 6:21 PM mruane--- via USRP-users <
usrp-users@lists.ettus.com> wrote:

OK here's some actual content for this thread outside of chatting about the
build system.

Pulling in features from the Xilinx fork to our X4xx kernel sources isn't
something we've had any kind of eye on, but it's interesting and we're
curious to see how this goes. It's something we'd like to see you succeed
in.

Currently our X4xx kernel sources are defined as follows: base branch:
git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git (see:
linux-x4xx.inc
https://github.com/EttusResearch/meta-ettus/blob/kirkstone/meta-ettus-bsp/recipes-kernel/linux/linux-x4xx.inc),
commit: e97bd1e03e6ef58ec47ee7f085f8c14ed6329cf7 (see linux-x4xx_5.10.bb
https://github.com/EttusResearch/meta-ettus/blob/kirkstone/meta-ettus-bsp/recipes-kernel/linux/linux-x4xx_5.10.bb),
patches: meta-ettus-bsp/recipes-kernel/linux/linux-x4xx
https://github.com/EttusResearch/meta-ettus/tree/kirkstone/meta-ettus-bsp/recipes-kernel/linux/linux-x4xx

One option we could discuss is if it were easier for folks if we based our
kernel on linux-xlnx instead of mainline (no promises or plans here, just
thinking out loud). Not everyone has Piotr-level expertise, and if this
would make things easier for folks, that could be something we do going
forward.

Whichever way we go -- of course you need to pull in patches from the right
kernel version. If that's something people struggle with, maybe we can make
that clearer? Just fishing for thoughts and/or suggestions here!

Cheers,

Martin

On Thu, Oct 3, 2024 at 1:51 PM perper@o2.pl wrote:

Hi Martin,

Thanks for all of the input.  I do like the UHD build system.  Make seems cryptic and convoluted at first glance, but it starts making sense pretty quickly, even if you don’t understand some of the more nuanced blocks.  I agree with your assessment of the other build systems, and I never really could put my finger on what it was that kept me from buying in.  I think it was our discussion with Marcus that gave me a clue as to what it might be.  Up to this point, I’ve never been particularly “for” or “against” Make; it is simply something that is pretty much everywhere.  The other build systems don’t have that.  More people are familiar with Make; it is installed by default on every Linux distribution I can think of, and I can pretty much take my code to any Linux machine with Vivado and build it.

Hahaha  I have to admit, I haven’t run rfnoc_image_builder yet, since I haven’t changed anything in the data-path.  To be honest, I don’t have much experience with RFNoC because I always design the data-path from scratch depending on what’s needed by the processing blocks and the larger system.  Not that I don’t re-use modules; it’s simply that I enjoy FIFOs, gearboxes, managing clock domains, flow-control, loopback, timing constraints, etc.  Is it weird that I’ve never felt like I wished there were something that could just take care of that for me?  Hahaha  Maybe. :-)

There really wasn’t anything that annoyed me…which is rare. :-)  It did take me a bit to figure out how to get the new bitfile and device tree into the rootfs build system, but I think I figured it out well enough.  What might help people is maybe adding an environment variable (or some similar mechanism) that both build tools use to point to the UHD build output (usrp_x410_fpga_xxx.bit and usrp_x410_fpga_xxx.dts).  The UHD build could export them, and they could be imported from there by the KAS config or in a meta-ettus recipe.  Maybe that’s similar to how it works already?  It just wasn’t obvious to me, so I modified our KAS config yaml, uhd-fpga-images.inc, and uhd-fpga-images_%.bbappend.

The reason I suggest that is because anyone that may be comfortable using the PetaLinux tools, but hasn’t yet experienced the “joy” (terror?) of their first pure Yocto experience is used to an export_hardware style relation ship between Vivado and PetaLinux that simplifies getting the PL device tree and bitfile into the image.

Mike

Hi Martin,

:-)  You caught me daydreaming about automating the tedious debug I’m about to start. :-)

Hi Martin,

Thanks for the recipe links.  As of yesterday, I figured that the linux-x4xx files were the place to start.  The files linux-usrp_5.10.bb and linux-yocto_5.2.* got me turned around at first, but I found my way out eventually. :-)

I don’t usually do much with the kernel, except use menuconfig to change the occasional CONFIG_ parameter, because the linux-xlnx kernel has most of the Xilinx-related configs that you would need to drive Xilinx IP in the PL.  I’m hoping that I can add a few required CONFIG_ parameters, and a few patches to get there, but so far that seems unlikely.

My concerns mostly arise from not knowing what to do when the Xilinx-specific parameters that I typically set in the linux-xlnx kernel config do not exist in the mainline.  For example, when adding the Vitis-AI libraries and the DPU driver to the build, it is necessary to set CONFIG_XILINX_DPU=y in the kernel config.  But when I add that line to x410_defconfig, and try to build an image, I get the following alert:

[INFO]: the following symbols were not found in the active configuration:
     - CONFIG_XILINX_DPU

The line, “not found in the current configuration”, makes it sounds like a KCONFIG-ish issue.  That means I have a little ‘fancy book-learnin’ to do to fix this one. :-) The build was successful, but I highly doubt that it did what I intended.  This isn’t a catastrophe…it just means that I have a little work ahead of me, and a good bit to learn.

Switching to the linux-xlnx kernel sounds like a great idea, but I don’t truly have a handle yet on the exact differences between the two kernels.  I’m not sure how to answer that in a way that takes all of your customer base into account.  So, from the selfish perspective of an FPGA developer that has recently learned to navigate the Yocto build system, it would presumably simplify things if the UHD kernel were based on linux-xlnx (especially if menuconfig were set up and available), because all of the Xilinx documentation and knowledge-base info would be relevant in addition to the Ettus documentation.

With that said, I don’t know what the implications of that move would be for your more Yocto savvy customers.

There’s a good point in there.  Modifying an FPGA design and building a custom embedded Linux distribution are non-trivial endeavors.  Ettus has simplified customization of some of the most difficult parts by including abstractions like RFNoC and gnuradio, and a Make-based build system.  But as soon as we begin customizing the FPGA design or rootfs, we will always need to make sure that we pull the right patches from the correct kernel version, or make sure that our PL customizations make their way into the image, etc.  I think that expanding the documentation (wiki, knowledge-base, etc) to be a little more in-depth for some topics would be helpful for developers of all levels.  For example, even if you don’t base the UHD kernel on lilnux-xlnx, perhaps documenting how to make a few customizations for common Xilinx additions would be extremely helpful.  The same would be true for making mods to the FPGA design that change the device tree.

Just a couple of thoughts.  It’ll take me a few days to better understand the kernel stuff.  Once I start making changes and debugging, I’ll add some of the progress and problems to the thread.

Martin Braun wrote: