Distribution Support Status¶
We endeavour to support popular Linux distributions on Ten64 boards as best as possible, however, given the newness of the hardware, interoperability issues may arise.
To provide the best possible out of box experience, we provide pre-customized and tested Images for some distributions, using our mainline kernel patch set. We may also post scripts that can 'patch' upstream images so they boot.
Generally, distributions with kernel's earlier than 5.0 will have no support for the DPAA2 network hardware.
Current status summary¶
Working on 'bare metal'¶
openSuSE Leap 15.2 and later.
SuSE Enterprise Linux / SLES certification is also planned.
Debian (with Traverse Debian Image / mainline kernel)
Ubuntu 20.04 and 20.10 (driver installation required, see section below)
These can be installed quickly using the bare metal appliance store.
* Criteria for working: boots from NVMe drive and working network interfaces.
Additional board support packages are available to implement missing functions such as SFP+ control, SFP activity LEDs and sensors.
Planned support (via patch / support script)¶
These do not currently run on the Ten64 due to bootloader and/or kernel issues.
We will supply a 'patch' script that will work around the issue (for example, installing a mainline kernel package with DPAA2 drivers enabled).
You can run them inside a virtual machine in the meantime.
|Distribution||Kernel||Status||Alternatives or Workarounds||Upstream bugs/changes (if any)|
|Debian Buster||4.19||No PCIe, no network. Needs IOMMU backport||Traverse Debian Image||Pending Debian bug #948576 for future Debian releases/testing/unstable|
|Debian Bullseye (testing)||5.8||As of 2020-10-07, no PCIe, but boots from a USB drive with working network support (IOMMU bypass argument needed)||Edit GRUB configuration before first boot||Debian bug #971777 for PCIe controller|
|Fedora 33||5.8/5.9||Boots, but need to manipulate image first (replace default EFI loader with grub). Kernel commandline arguments need to be changed to provide serial console. Kernel 5.9 has issues with DPAA2 Ethernet||Replace default BOOTAA64.EFI with GRUB, add arm-smmu.disable_bypass=n to command line. See below for full steps|
Ubuntu 20.04 and 20.10 steps¶
The Ubuntu "cloud images" come with a slimmed down kernel which does not have the DPAA2 ethernet drivers by default. You will need to install the "extra" modules package if the Ubuntu image you are using does not contain the DPAA2 Ethernet and USB3 drivers.
Install the 'extra modules' package for your kernel version
If your kernel version is:
root@ubuntu:~# uname -a Linux ubuntu 5.8.0-31-generic #33-Ubuntu SMP Mon Nov 23 18:51:30 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
You will need to install
root@ubuntu:~# apt-get install linux-modules-extra-5.8.0-31-generic
(Note: Make sure you have the latest available kernel package first - Ubuntu will not upgrade you automatically)
By default, Ubuntu will hide the GRUB menu, removing your ability to interrupt the kernel boot and/or see any relevant error messages.
You can fix this by changing the variables
root@ubuntu:~# head /etc/default/grub # If you change this file, run 'update-grub' afterwards to update # /boot/grub/grub.cfg. # For full documentation of the options in this file, see: # info -f grub -n 'Simple configuration' GRUB_DEFAULT=0 GRUB_TIMEOUT_STYLE=menu GRUB_TIMEOUT=20 GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian` GRUB_CMDLINE_LINUX_DEFAULT="earlycon" GRUB_CMDLINE_LINUX=""
update-grubto update the GRUB configuration and add any new kernels.
This is not needed on Ten64 as we use EFI+GRUB to boot.
flash-kernelis not removed, Ubuntu tries to invoke it during a kernel package upgrade/install, causing apt to exit with an error.
root@ubuntu:~# apt-get remove flash-kernel
Please note: The first boot under Ubuntu 20.04 will panic and restart due to lack of NVMe driver.
This is because Ubuntu 20.04 first tries to boot without an initrd. If the first boot does not complete, GRUB will then subsequently use an initrd for subsequent boots, which will complete without issue.
A couple of changes are needed for Fedora 33:
Use GRUB directly instead of via SHIM:
mount /dev/mapper/fedora_fedora-root /mnt/fedora/ mount /dev/nvme0n1p2 /mnt/fedora/boot mount /dev/nvme0n1p1 /mnt/fedora/boot/efi cp /mnt/fedora/boot/efi/EFI/fedora/grubaa64.efi /mnt/fedora/boot/efi/EFI/BOOT/BOOTAA64.efi
Change the kernel command line:
Fedora aggressively tries to use a framebuffer/video tty as the system console, which does not exist on the Ten64.
The kernel arguments need to be:
root=/dev/mapper/fedora_fedora-root ro rd.lvm.lv=fedora_fedora/root console=ttyS0,115200 earlycon arm-smmu.disable_bypass=n
Specifically, you need to change everything after
rd.lvm.lvin the default install.
This can be changed in
[root@localhost EFI]# grep GRUB_CMDLINE_LINUX /etc/default/grub GRUB_CMDLINE_LINUX="rd.lvm.lv=fedora_fedora/root console=ttyS0,115200 earlycon arm-smmu.disable_bypass=n"
And you can regenerate a GRUB configuration:
grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg
The 5.9 kernel offered as an update has issues with it's DPAA2 driver (as of 2020-12-07). You can use versionlock to stop it from being installed:
dnf install 'dnf-command(versionlock)' dnf versionlock kernel-5.8.15-301.fc33 kernel-core-5.8.15-301.fc33 kernel-modules-5.8.15-301.fc33
General compatibility notes:¶
Kernel 5.1 and later - IOMMU bypass / fault on boot¶
Kernel 5.1 disabled IOMMU bypass on ARM64, but the DPAA2 drivers currently do not function without it.
arm-smmu.disable_bypass=n to the kernel command line.
Our images are configured to add this to the command line by default.
- See this NXP community post for more information and current status of upstream fixes.
Some distributions have a shim EFI loader (such as Red Hat's shim) that requires more advanced UEFI functions than currently implemented by U-Boot's EFI subsystem.
Workaround: Move the GRUB EFI binary into it's place.
For example, on Fedora 31:
mv /boot/efi/fedora/grubaa64.efi /boot/efi/EFI/bootaa64.efi
Issues with the shim loader should go away once UEFI Secure Boot and variable storage is implemented in U-Boot - see our roadmap for details.
Some distributions specify 'quiet' mode or have a splash screen, which tends to work poorly with serial consoles. If there is a kernel panic on boot, you may not be able to see it before the kernel triggers a reboot.
Recommendation: Replace any 'quiet' and/or 'splash' arguments on the kernel command line with:
Usually, there is no need to specify console=ttyS0 etc. as the device tree has been set up to link this automatically (via linux,stdout-path). However, note that the LS1088A has a conventional PC-style 16550 UART rather than the ARM PrimeCell UART (ttyAMA0) found on most ARM boards - you can safely remove any 'console=' references on the kernel command line.
For non-systemd distributions, you may need to set ttyS0 as the console in
USB3/DWC splat on probe.¶
You may see an error similar to this:
[ 7.514992] dwc3 3100000.usb3: Failed to get clk 'ref': -2 [ 7.522173] dwc3 3110000.usb3: Failed to get clk 'ref': -2
This is harmless, the USB3 ports will still work.