Hi,
I'm on embedded imx8 mm platform and trying to order services such that ptp4l (LinuxPTP) is started after a bond is created between two DSA network interfaces on my Microchip Ethernet Switch.
No matter what I try with BindsTo, Wants=, Requires=, Before=, After=, when the board boots and I watch the console output I see systemd start the ptp4l service before the bond is actually up which results in ptp4l failing to start.
I've band-aided the issue by putting a Restart=yes and RestartSec=1 in the service but feel like this is a hack.
As you can see below, the bond is in the process of coming up (you can see enslaving messages regarding lan1 and lan2) but it is not all the way up until after the login prompt ... at which point the service starting ptp4l (the Sync Microchip PHC with PTP Grand Master Clock msg) already failed. From timestamps it looks like it's taking 4 seconds or so for bond to come up:
[ OK ] Started Network Service.
Starting Network Name Resolution...
[ OK ] Started Rauc Good-marking Service.
[ 4.098258] imx-sdma 302c0000.dma-controller: firmware found.
[ 4.104259] imx-sdma 302c0000.dma-controller: loaded firmware 4.5
[ OK ] Started Network Name Resolution[ 4.135108] imx-sdma 302b0000.dma-controller: firmware found.
.
[ OK ] Reached targe[ 4.146220] imx-sdma 30bd0000.dma-controller: firmware found.
t Network.
[ OK ] Reached targe[ 4.159060] caam-snvs 30370000.caam-snvs: violation handlers armed - init state
t Network is Online.
[ OK ] Reached target Host and Network Name Lookups.
Starting Avahi mDNS/DNS-SD Stack...
Starting Enable ksz9567...
Starting The NGINX HTTP and reverse proxy server...
[ OK ] Started Enable ksz9567.
[ OK ] Started Avahi mDNS/DNS-SD Stack.
[ OK ] Found device /sys/subsystem/net/devices/lan2.
[ OK ] Started The NGINX HTTP and reverse proxy server.
[ 4.390005] ksz9477-switch 0-005f lan2: configuring for phy/gmii link mode
[ 4.402902] bond1: (slave lan2): Enslaving as a backup interface with a down link
[ OK ] Found device [ 4.544105] ksz9477-switch 0-005f lan1: configuring for phy/gmii link mode
/sys/subsystem/net/devices/lan1.
[ 4.555298] bond1: (slave lan1): Enslaving as a backup interface with a down link
[ OK ] Found device /sys/subsystem/net/devices/bond1.
Starting Save/Restore Sound Card State...
[ OK ] Started Save/Restore Sound Card State.
[ OK ] Reached target Sound Card.
[ 6.714002] random: crng init done
[ 6.717436] random: 7 urandom warning(s) missed due to ratelimiting
[ OK ] Started Load/Save Random Seed.
[ OK ] Started System Logger Daemon "default" instance.
[ OK ] Reached target Multi-User System.
Starting Sync Microchip PH…with PTP Grand Master Clock...
Starting Update UTMP about System Runlevel Changes...
[FAILED] Failed to start Sync Micro…C with PTP Grand Master Clock.
See 'systemctl status ptp4l.service' for details.
[ OK ] Started Update UTMP about System Runlevel Changes.
[ 7.659157] ksz9477-switch 0-005f lan1: Link is Up - 1Gbps/Full - flow control rx/tx
Poky (Yocto Project Reference Distro) 3.1.7 imx8mmevk ttymxc1
imx8mmevk login: [ 8.936988] bond1: (slave lan1): link status up, enabling it in 0 ms
[ 8.943510] bond1: (slave lan1): link status definitely up, 1000 Mbps full duplex
[ 8.951068] bond1: (slave lan1): making interface the new active one
[ 8.962012] device eth0 entered promiscuous mode
[ 8.966708] audit: type=1700 audit(1600598644.739:2): dev=eth0 prom=256 old_prom=0 auid=4294967295 uid=0 gid=0 ses=4294967295
[ 8.966908] bond1: active interface up!
[ 8.985999] IPv6: ADDRCONF(NETDEV_CHANGE): bond1: link becomes ready
Starting Network Name Resolution...
[ OK ] Started Rauc Good-marking Service.
[ 4.098258] imx-sdma 302c0000.dma-controller: firmware found.
[ 4.104259] imx-sdma 302c0000.dma-controller: loaded firmware 4.5
[ OK ] Started Network Name Resolution[ 4.135108] imx-sdma 302b0000.dma-controller: firmware found.
.
[ OK ] Reached targe[ 4.146220] imx-sdma 30bd0000.dma-controller: firmware found.
t Network.
[ OK ] Reached targe[ 4.159060] caam-snvs 30370000.caam-snvs: violation handlers armed - init state
t Network is Online.
[ OK ] Reached target Host and Network Name Lookups.
Starting Avahi mDNS/DNS-SD Stack...
Starting Enable ksz9567...
Starting The NGINX HTTP and reverse proxy server...
[ OK ] Started Enable ksz9567.
[ OK ] Started Avahi mDNS/DNS-SD Stack.
[ OK ] Found device /sys/subsystem/net/devices/lan2.
[ OK ] Started The NGINX HTTP and reverse proxy server.
[ 4.390005] ksz9477-switch 0-005f lan2: configuring for phy/gmii link mode
[ 4.402902] bond1: (slave lan2): Enslaving as a backup interface with a down link
[ OK ] Found device [ 4.544105] ksz9477-switch 0-005f lan1: configuring for phy/gmii link mode
/sys/subsystem/net/devices/lan1.
[ 4.555298] bond1: (slave lan1): Enslaving as a backup interface with a down link
[ OK ] Found device /sys/subsystem/net/devices/bond1.
Starting Save/Restore Sound Card State...
[ OK ] Started Save/Restore Sound Card State.
[ OK ] Reached target Sound Card.
[ 6.714002] random: crng init done
[ 6.717436] random: 7 urandom warning(s) missed due to ratelimiting
[ OK ] Started Load/Save Random Seed.
[ OK ] Started System Logger Daemon "default" instance.
[ OK ] Reached target Multi-User System.
Starting Sync Microchip PH…with PTP Grand Master Clock...
Starting Update UTMP about System Runlevel Changes...
[FAILED] Failed to start Sync Micro…C with PTP Grand Master Clock.
See 'systemctl status ptp4l.service' for details.
[ OK ] Started Update UTMP about System Runlevel Changes.
[ 7.659157] ksz9477-switch 0-005f lan1: Link is Up - 1Gbps/Full - flow control rx/tx
Poky (Yocto Project Reference Distro) 3.1.7 imx8mmevk ttymxc1
imx8mmevk login: [ 8.936988] bond1: (slave lan1): link status up, enabling it in 0 ms
[ 8.943510] bond1: (slave lan1): link status definitely up, 1000 Mbps full duplex
[ 8.951068] bond1: (slave lan1): making interface the new active one
[ 8.962012] device eth0 entered promiscuous mode
[ 8.966708] audit: type=1700 audit(1600598644.739:2): dev=eth0 prom=256 old_prom=0 auid=4294967295 uid=0 gid=0 ses=4294967295
[ 8.966908] bond1: active interface up!
[ 8.985999] IPv6: ADDRCONF(NETDEV_CHANGE): bond1: link becomes ready
My ptp4l.service looks like:
cat ptp4l.service
[Unit]
Description=Sync Microchip PHC with PTP Grand Master Clock
Requires=network-online.target multi-user.target
BindsTo=sys-subsystem-net-devices-bond1.device sys-subsystem-net-devices-lan1.device sys-subsystem-net-devices-lan2.device multi-user.target
After=sys-subsystem-net-devices-bond1.device sys-subsystem-net-devices-lan1.device sys-subsystem-net-devices-lan2.device multi-user.target
[Service]
Type=exec
#NotifyAccess=all
ExecStart=/usr/bin/ptp4l -f /etc/linuxptp/ptp4l.conf_e2e_one_step_g8275.2 -s -i bond1
Restart=on-failure
RestartSec=1
[Install]
WantedBy=multi-user.target
[Unit]
Description=Sync Microchip PHC with PTP Grand Master Clock
Requires=network-online.target multi-user.target
BindsTo=sys-subsystem-net-devices-bond1.device sys-subsystem-net-devices-lan1.device sys-subsystem-net-devices-lan2.device multi-user.target
After=sys-subsystem-net-devices-bond1.device sys-subsystem-net-devices-lan1.device sys-subsystem-net-devices-lan2.device multi-user.target
[Service]
Type=exec
#NotifyAccess=all
ExecStart=/usr/bin/ptp4l -f /etc/linuxptp/ptp4l.conf_e2e_one_step_g8275.2 -s -i bond1
Restart=on-failure
RestartSec=1
[Install]
WantedBy=multi-user.target
In my .network file I specify that bond1 is required to for being "online":
[Match]
Name=bond1
[Network]
#BindCarrier=eth0
DHCP=yes
#Address=192.168.0.4/24
[Link]
RequiredForOnline=yes
In .system file I tried all I know to ensure the required interfaces were created before starting ptp4l in attempt to give bonding enough time to finish but binding to things like sys-subsystem-net-devices-bond1.device wasn't enough.
Is it also possible to use carrier state in .service file?
I see sys/devices/virtual/net/bond1/carrier but not sure how to only attempt to start my ptp4l service after carrier state is "1".
I welcome your ideas and suggestions on how to start a service after a bond interface is really up.
Regards,
Brian
