Skip to content

PHY Management / Ethernet link status Modes

Management modes overview

On the Ten64, there are two ways the Ethernet ports can be managed:

  • Legacy mode is where the DPAA2 MC firmware manages the status of the ethernet links internally.

    For the 1000BASE-T ports, this means the operating system will always see them as "up" regardless of the actual physical link status.

    The link status of SFP+ ports will be reported to the operating system, but the operating system will need to manage the SFP+ control lines from userspace.

  • Managed mode is where the operating system will handle communication between the processor and Ethernet PHYs, and report link up/down events to the MC firmware.

    Managed mode also provides full SFP+ management under Linux, such as control of the SFP+ I/O signals, reading diagnostics via ethtool and temperature sensors via hwmon (where applicable).

Managed mode is the default mode Ten64 units are shipped in and requires support from the operating system, which has been introduced from Linux kernel 5.8 onwards (as part of the Phylink subsystem). Some older kernels/distributions (e.g Ubuntu 18.04, openSUSE Leap 15.2) will fail to enable Ethernet ports when in managed mode.

Additionally, the Linux mainline kernel did not have full support for SFP+ management on DPAA2 platforms until version 5.16, see "Kernel patches required for SFP support" for more information.

You can change the management mode for both the copper ports (1000BASE-T) and SFP+ cages using environment variables in U-Boot, and the bare metal appliance store will prompt you to change the management modes to suit the operating system you intend to deploy.

Linux Kernel Version Gigabit/Copper ports managed? SFP+ ports managed?
Before 5.8 ❌
NO		 ❌
NO
After 5.8
(most mainstream distributions)		 ✅
YES		 ❌
NO
After 6.2
5.8 with SFP+ support patches
(Traverse OpenWrt, kernel, μVirt, VyOS etc.)		 ✅
YES		 ✅
YES

Libraries / applications such as DPDK that communicate with the DPAA2 management complex directly, and other use cases where there is no direct communication with the PHY (such as VFIO passthrough) also need to be used in legacy mode at this time.

(In these cases, you can also modify the DPAA2 configuration so some ports are managed and some ports are not)

Changing management mode

There are two environment variables in U-Boot that can control the management behavior:

  • gbemode will control the management status of the copper/1000BASE-T ports
  • sfpmode will control the management status of the SFP+ ports

This function was introduced in the Ten64 firmware release 0.8.5, which was released after the first Ten64 retail shipments. You will need to update your firmware to use this function.

Changing to legacy mode

After setting the gbemode and/or sfpmode variables, you will need to do a board reset.

This is because legacy mode affects the startup configuration of the DPAA2 management complex, which is booted just before you reach the U-Boot prompt:

setenv gbemode legacy
setenv sfpmode legacy
saveenv
reset

Changing to managed mode

To change to managed mode, simply delete the gbemode and/or sfpmode variables.

As above, you will need to reset the board for the changes to the effect.

env delete gbemode
env delete sfpmode
saveenv
reset

Technical notes

In legacy mode, the DPC is fixed-up at boot to change the MAC configuration from MAC_LINK_TYPE_PHY to MAC_LINK_TYPE_FIXED.

Setting the SFP+ into legacy mode will also disable the Linux sff,sfp bindings in the device tree that is passed to the operating system.