In my case, ZFS was failing to import a zpool because it was on a cloud persistent volume that was not physically attached to the machine. I guess network volumes become available later than expected in the bootup process.
Running systemctl status zfs-import-cache.service
after boot gave the following message:
● zfs-import-cache.service - Import ZFS pools by cache file Loaded: loaded (/lib/systemd/system/zfs-import-cache.service; enabled; vendor preset: enabled) Active: failed (Result: exit-code) since Tue 2021-09-07 18:37:28 UTC; 3 months 17 days ago Docs: man:zpool(8) Process: 780 ExecStart=/sbin/zpool import -c /etc/zfs/zpool.cache -aN (code=exited, status=1/FAILURE) Main PID: 780 (code=exited, status=1/FAILURE)Sep 07 18:37:26 ingress-zfs-2 systemd[1]: Starting Import ZFS pools by cache file...Sep 07 18:37:28 ingress-zfs-2 zpool[780]: cannot import 'data': no such pool or datasetSep 07 18:37:28 ingress-zfs-2 zpool[780]: Destroy and re-create the pool fromSep 07 18:37:28 ingress-zfs-2 zpool[780]: a backup source.Sep 07 18:37:28 ingress-zfs-2 systemd[1]: zfs-import-cache.service: Main process exited, code=exited, status=1/FAILURESep 07 18:37:28 ingress-zfs-2 systemd[1]: zfs-import-cache.service: Failed with result 'exit-code'.Sep 07 18:37:28 ingress-zfs-2 systemd[1]: Failed to start Import ZFS pools by cache file.
The solution was to patch the zfs-import-cache.service
service file to include the remote-fs.target
dependency:
[Unit]...After=remote-fs.target...
On Ubuntu 20.04, this file was located at: /etc/systemd/system/zfs-import.target.wants/zfs-import-cache.service
.
I think specifying After=remote-fs.target
is equivalent to using the _netdev
option in an /etc/fstab
file (see: https://unix.stackexchange.com/a/226453/78327).