Distribution Support Notes¶

See Does it run? for the current distribution compatibility table.

This page describes known distribution interoperability issues and the workarounds for them.

The Bare Metal Appliance Store will apply these fixes automatically based on manipulator scripts for the relevant distributions.

Generally, distributions with kernel's earlier than 5.0 will have no support for the DPAA2 network hardware.

A 5.10 or later kernel is generally recommended as these will have managed network mode support.

Detailed notes¶

Compatibility blockers¶

Distribution	Kernel	Status	Alternatives or Workarounds	Upstream bugs/changes (if any)
Debian Buster	4.19	No PCIe, no network. Needs IOMMU backport	Install a backport kernel or our kernel package	Pending Debian bug #948576 for future Debian releases/testing/unstable
Debian Bullseye (testing)	5.8	As of 2020-10-07, no PCIe, but boots from a USB drive with working network support (IOMMU bypass argument needed)	Edit GRUB configuration before first boot	Debian bug #971777 for PCIe controller
CentOS 8 / other RHEL 8 derivatives	4.18	Kernel too old	Install a newer kernel RPM (e.g from Fedora). We do not recommend running CentOS baremetal on Ten64 at this time. Consider running as a VM if you have a RHEL/CentOS only workload.
Fedora 33	5.8/5.9	Boots, but need to manipulate image first (replace default EFI loader with grub). Kernel commandline arguments need to be changed to provide serial console. Kernel 5.9 has issues with DPAA2 Ethernet	Replace default BOOTAA64.EFI with GRUB, add arm-smmu.disable_bypass=n to command line. See below for full steps

Ubuntu 20.04 and 20.10 steps¶

The Ubuntu "cloud images" come with a slimmed down kernel which does not have the DPAA2 ethernet drivers by default. You will need to install the "extra" modules package if the Ubuntu image you are using does not contain the DPAA2 Ethernet and USB3 drivers.

Install the 'extra modules' package for your kernel version

If your kernel version is:
```
root@ubuntu:~# uname -a
Linux ubuntu 5.8.0-31-generic #33-Ubuntu SMP Mon Nov 23 18:51:30 UTC 2020 aarch64 aarch64 aarch64 GNU/Linux
```
You will need to install linux-modules-extra-...-generic:

root@ubuntu:~# apt-get install linux-modules-extra-5.8.0-31-generic

(Note: Make sure you have the latest available kernel package first - Ubuntu will not upgrade you automatically)

By default, Ubuntu will hide the GRUB menu, removing your ability to interrupt the kernel boot and/or see any relevant error messages.

You can fix this by changing the variables GRUB_TIMEOUT_STYLE, GRUB_TIMEOUT and GRUB_CMDLINE_LINUX_DEFAULT in /etc/default/grub:

root@ubuntu:~# head /etc/default/grub
# If you change this file, run 'update-grub' afterwards to update
# /boot/grub/grub.cfg.
# For full documentation of the options in this file, see:
#   info -f grub -n 'Simple configuration'

GRUB_DEFAULT=0
GRUB_TIMEOUT_STYLE=menu
GRUB_TIMEOUT=20
GRUB_DISTRIBUTOR=`lsb_release -i -s 2> /dev/null || echo Debian`
GRUB_CMDLINE_LINUX_DEFAULT="earlycon"
GRUB_CMDLINE_LINUX=""

Run update-grub to update the GRUB configuration and add any new kernels.

Remove the flash-kernel package

This is not needed on Ten64 as we use EFI+GRUB to boot.

If flash-kernel is not removed, Ubuntu tries to invoke it during a kernel package upgrade/install, causing apt to exit with an error.

root@ubuntu:~# apt-get remove flash-kernel

Please note: The first boot under Ubuntu 20.04 will panic and restart due to lack of NVMe driver.

This is because Ubuntu 20.04 first tries to boot without an initrd. If the first boot does not complete, GRUB will then subsequently use an initrd for subsequent boots, which will complete without issue.

Fedora 33¶

A couple of changes are needed for Fedora 33:

Use GRUB directly instead of via SHIM:

mount /dev/mapper/fedora_fedora-root /mnt/fedora/
mount /dev/nvme0n1p2 /mnt/fedora/boot
mount /dev/nvme0n1p1 /mnt/fedora/boot/efi
cp /mnt/fedora/boot/efi/EFI/fedora/grubaa64.efi /mnt/fedora/boot/efi/EFI/BOOT/BOOTAA64.efi

Change the kernel command line:

Fedora aggressively tries to use a framebuffer/video tty as the system console, which does not exist on the Ten64.

The kernel arguments need to be:

root=/dev/mapper/fedora_fedora-root ro rd.lvm.lv=fedora_fedora/root console=ttyS0,115200 earlycon arm-smmu.disable_bypass=n

Specifically, you need to change everything after rd.lvm.lv in the default install.

This can be changed in /etc/default/grub:

[root@localhost EFI]# grep GRUB_CMDLINE_LINUX /etc/default/grub
GRUB_CMDLINE_LINUX="rd.lvm.lv=fedora_fedora/root console=ttyS0,115200 earlycon arm-smmu.disable_bypass=n"

And you can regenerate a GRUB configuration:

grub2-mkconfig -o /boot/efi/EFI/fedora/grub.cfg

The 5.9 kernel offered as an update has issues with it's DPAA2 driver (as of 2020-12-07). You can use versionlock to stop it from being installed:
```
dnf install 'dnf-command(versionlock)' 
dnf versionlock kernel-5.8.15-301.fc33 kernel-core-5.8.15-301.fc33 kernel-modules-5.8.15-301.fc33
```

General compatibility notes:¶

Kernel 5.1 and later - IOMMU bypass / fault on boot¶

Kernel 5.1 disabled IOMMU bypass on ARM64, but the DPAA2 drivers currently do not function without it. Workaround: Add arm-smmu.disable_bypass=n to the kernel command line.

The bare metal appliance store will modify the default kernel command line to add this setting by default (where needed).

See this NXP community post for more information and current status of upstream fixes.

EFI loaders¶

Some distributions have a shim EFI loader (such as Red Hat's shim) that requires more advanced UEFI functions than currently implemented by U-Boot's EFI subsystem.

Workaround: Move the GRUB EFI binary into it's place.

For example, on Fedora 31:

mv /boot/efi/fedora/grubaa64.efi /boot/efi/EFI/bootaa64.efi

Issues with the shim loader should go away once UEFI Secure Boot and variable storage is implemented in U-Boot - see our roadmap for details.

Serial console¶

Some distributions specify 'quiet' mode or have a splash screen, which tends to work poorly with serial consoles. If there is a kernel panic on boot, you may not be able to see it before the kernel triggers a reboot.

Recommendation: Replace any 'quiet' and/or 'splash' arguments on the kernel command line with:

earlycon

Usually, there is no need to specify console=ttyS0 etc. as the device tree has been set up to link this automatically (via linux,stdout-path). However, note that the LS1088A has a conventional PC-style 16550 UART rather than the ARM PrimeCell UART (ttyAMA0) found on most ARM boards - you can safely remove any 'console=' references on the kernel command line.

Note

For non-systemd distributions, you may need to set ttyS0 as the console in /etc/inittab

USB3/DWC splat on probe.¶

You may see an error similar to this:

[    7.514992] dwc3 3100000.usb3: Failed to get clk 'ref': -2
[    7.522173] dwc3 3110000.usb3: Failed to get clk 'ref': -2

This is harmless, the USB3 ports will still work.