Re: xfs: system fails to boot up due to Internal error xfs_trans_cancel

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Sat, Feb 18, 2023 at 12:47:48PM +0530, shrikanth hegde wrote:
> 
> 
> On 2/17/23 10:23 PM, Darrick J. Wong wrote:
> > On Fri, Feb 17, 2023 at 04:45:12PM +0530, shrikanth hegde wrote:
> >> We are observing panic on boot upon loading the latest stable tree(v6.2-rc4) in 
> >> one of our systems. System fails to come up. System was booting well 
> >> with v5.17, v5.19 kernel. We started seeing this issue when loading v6.0 kernel.
> >>
> >> Panic Log is below.
> >> [  333.390539] ------------[ cut here ]------------
> >> [  333.390552] WARNING: CPU: 56 PID: 12450 at fs/xfs/xfs_inode.c:1839 xfs_iunlink_lookup+0x58/0x80 [xfs]
> > 
> > Hmm, ok, so this is the same if (WARN_ON_ONCE(!ip || !ip->i_ino)) line
> > in xfs_iunlink_lookup that I've been bonking my head on the past
> > several days.  333 seconds uptime, so I guess this is a pretty recent
> > mount.  You didn't post a full dmesg, so I can only assume there weren't
> > any *other* obvious complaints from XFS when the fs was mounted...
> > 
> 
> Darrick, Dave. Thank you for taking a look at this. sorry i didnt paste the full log.
> Please find the full dmesg below. it was kexec from 5.17-rc2 to 6.2-rc8 
> 
> 
>          Starting Reboot via kexec...
> [   71.134231] printk: systemd-shutdow: 37 output lines suppressed due to ratelimiting
> [   71.158255] systemd-shutdown[1]: Syncing filesystems and block devices.
> [   71.160419] systemd-shutdown[1]: Sending SIGTERM to remaining processes...
> [   71.214007] systemd-shutdown[1]: Sending SIGKILL to remaining processes...
> [   71.233182] systemd-shutdown[1]: Unmounting file systems.
> [   71.233845] [2287]: Remounting '/home' read-only in with options 'attr2,inode64,logbufs=8,logbsize=32k,noquota'.
> [   71.234873] [2288]: Unmounting '/home'.
> [   71.237859] XFS (nvme0n1p1): Unmounting Filesystem
> [   71.524111] [2289]: Remounting '/' read-only in with options 'attr2,inode64,logbufs=8,logbsize=32k,noquota'.
> [   71.810094] systemd-shutdown[1]: All filesystems unmounted.
> [   71.810103] systemd-shutdown[1]: Deactivating swaps.
> [   71.810121] systemd-shutdown[1]: All swaps deactivated. pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=systemd-remount-fs comm="systemd" exe="/usr/li[   71.810126] systemd-shutdown[1]: Detaching loop devices.'
> [   71.810227] systemd-shutdown[1]: All loop devices detached.
> [   71.810232] systemd-shutdown[1]: Detaching DM devices.
> [   71.932097] printk: shutdown: 8 output lines suppressed due to ratelimiting
> [   71.954484] dracut Warning: Killing all remaining processes
> dracut Warning: Killing all remaining processes
> [   72.001360] XFS (dm-0): Unmounting Filesystem
> [   72.393816] dracut Warning: Unmounted /oldroot.
> [   72.409893] dracut: Disassembling device-mapper devices
> [   72.527853] Removing IBM Power 842 compression device
> [   78.084038] kexec_core: Starting new kernel
> [   78.104094] kexec: waiting for cpu 26 (physical 26) to enter 1 state
> [   78.104102] kexec: waiting for cpu 87 (physical 87) to enter 1 state
> [   78.104109] kexec: waiting for cpu 1 (physical 1) to enter 2 state
> [   78.104248] kexec: waiting for cpu 2 (physical 2) to enter 2 state
> [   78.104275] kexec: waiting for cpu 3 (physical 3) to enter 2 state
> [   78.104289] kexec: waiting for cpu 4 (physical 4) to enter 2 state
> [   78.104303] kexec: waiting for cpu 5 (physical 5) to enter 2 state
> [   78.104315] kexec: waiting for cpu 8 (physical 8) to enter 2 state
> [   78.104352] kexec: waiting for cpu 48 (physical 48) to enter 2 state
> [   78.104357] kexec: waiting for cpu 49 (physical 49) to enter 2 state
> [   78.278072] kexec: Starting switchover sequence.
> I'm in purgatory
> [    0.000000] radix-mmu: Page sizes from device-tree:
> [    0.000000] radix-mmu: Page size shift = 12 AP=0x0
> [    0.000000] radix-mmu: Page size shift = 16 AP=0x5
> [    0.000000] radix-mmu: Page size shift = 21 AP=0x1
> [    0.000000] radix-mmu: Page size shift = 30 AP=0x2
> [    0.000000] Activating Kernel Userspace Access Prevention
> [    0.000000] Activating Kernel Userspace Execution Prevention
> [    0.000000] radix-mmu: Mapped 0x0000000000000000-0x0000000002600000 with 2.00 MiB pages (exec)
> [    0.000000] radix-mmu: Mapped 0x0000000002600000-0x0000004b00000000 with 2.00 MiB pages
> [    0.000000] lpar: Using radix MMU under hypervisor
> [    0.000000] Linux version 6.2.0-rc8ssh (root@xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx) (gcc (GCC) 8.5.0 20210514 (Red Hat 8.5.0-10), GNU ld version 2.30-113.el8) #35 SMP Sat Feb 18 00:41:56 EST 2023
> [    0.000000] Found initrd at 0xc0000000061d0000:0xc00000000ac38d3b
> [    0.000000] Hardware name: IBM,9080-HEX POWER10 (raw) 0x800200 0xf000006 of:IBM,FW1010.22 (NH1010_122) hv:phyp pSeries
> [    0.000000] printk: bootconsole [udbg0] enabled
> [    0.000000] Partition configured for 96 cpus.
> [    0.000000] CPU maps initialized for 8 threads per core
> [    0.000000] numa: Partition configured for 32 NUMA nodes.
> [    0.000000] -----------------------------------------------------
> [    0.000000] phys_mem_size     = 0x4b00000000
> [    0.000000] dcache_bsize      = 0x80
> [    0.000000] icache_bsize      = 0x80
> [    0.000000] cpu_features      = 0x000c00eb8f5f9187
> [    0.000000]   possible        = 0x000ffbfbcf5fb187
> [    0.000000]   always          = 0x0000000380008181
> [    0.000000] cpu_user_features = 0xdc0065c2 0xaef60000
> [    0.000000] mmu_features      = 0xbc007641
> [    0.000000] firmware_features = 0x000005bfc55bfc57
> [    0.000000] vmalloc start     = 0xc008000000000000
> [    0.000000] IO start          = 0xc00a000000000000
> [    0.000000] vmemmap start     = 0xc00c000000000000
> [    0.000000] -----------------------------------------------------
> [    0.000000] numa:   NODE_DATA [mem 0x4afe663780-0x4afe66aeff]
> [    0.000000] rfi-flush: fallback displacement flush available
> [    0.000000] count-cache-flush: hardware flush enabled.
> [    0.000000] link-stack-flush: software flush enabled.
> [    0.000000] stf-barrier: eieio barrier available
> [    0.000000] PPC64 nvram contains 15360 bytes
> [    0.000000] barrier-nospec: using ORI speculation barrier
> [    0.000000] Zone ranges:
> [    0.000000]   Normal   [mem 0x0000000000000000-0x0000004affffffff]
> [    0.000000]   Device   empty
> [    0.000000] Movable zone start for each node
> [    0.000000] Early memory node ranges
> [    0.000000]   node   2: [mem 0x0000000000000000-0x0000004affffffff]
> [    0.000000] Initializing node 0 as memoryless
> [    0.000000] Initmem setup node 0 as memoryless
> [    0.000000] Initializing node 1 as memoryless
> [    0.000000] Initmem setup node 1 as memoryless
> [    0.000000] Initmem setup node 2 [mem 0x0000000000000000-0x0000004affffffff]
> [    0.000000] Initializing node 3 as memoryless
> [    0.000000] Initmem setup node 3 as memoryless
> [    0.000000] Initializing node 4 as memoryless
> [    0.000000] Initmem setup node 4 as memoryless
> [    0.000000] Initializing node 5 as memoryless
> [    0.000000] Initmem setup node 5 as memoryless
> [    0.000000] Initializing node 6 as memoryless
> [    0.000000] Initmem setup node 6 as memoryless
> [    0.000000] Initializing node 7 as memoryless
> [    0.000000] Initmem setup node 7 as memoryless
> [    0.000000] Initializing node 8 as memoryless
> [    0.000000] Initmem setup node 8 as memoryless
> [    0.000000] Initializing node 9 as memoryless
> [    0.000000] Initmem setup node 9 as memoryless
> [    0.000000] Initializing node 10 as memoryless
> [    0.000000] Initmem setup node 10 as memoryless
> [    0.000000] Initializing node 11 as memoryless
> [    0.000000] Initmem setup node 11 as memoryless
> [    0.000000] Initializing node 12 as memoryless
> [    0.000000] Initmem setup node 12 as memoryless
> [    0.000000] Initializing node 13 as memoryless
> [    0.000000] Initmem setup node 13 as memoryless
> [    0.000000] Initializing node 14 as memoryless
> [    0.000000] Initmem setup node 14 as memoryless
> [    0.000000] Initializing node 15 as memoryless
> [    0.000000] Initmem setup node 15 as memoryless
> [    0.000000] Initializing node 16 as memoryless
> [    0.000000] Initmem setup node 16 as memoryless
> [    0.000000] Initializing node 17 as memoryless
> [    0.000000] Initmem setup node 17 as memoryless
> [    0.000000] Initializing node 18 as memoryless
> [    0.000000] Initmem setup node 18 as memoryless
> [    0.000000] Initializing node 19 as memoryless
> [    0.000000] Initmem setup node 19 as memoryless
> [    0.000000] Initializing node 20 as memoryless
> [    0.000000] Initmem setup node 20 as memoryless
> [    0.000000] Initializing node 21 as memoryless
> [    0.000000] Initmem setup node 21 as memoryless
> [    0.000000] Initializing node 22 as memoryless
> [    0.000000] Initmem setup node 22 as memoryless
> [    0.000000] Initializing node 23 as memoryless
> [    0.000000] Initmem setup node 23 as memoryless
> [    0.000000] Initializing node 24 as memoryless
> [    0.000000] Initmem setup node 24 as memoryless
> [    0.000000] Initializing node 25 as memoryless
> [    0.000000] Initmem setup node 25 as memoryless
> [    0.000000] Initializing node 26 as memoryless
> [    0.000000] Initmem setup node 26 as memoryless
> [    0.000000] Initializing node 27 as memoryless
> [    0.000000] Initmem setup node 27 as memoryless
> [    0.000000] Initializing node 28 as memoryless
> [    0.000000] Initmem setup node 28 as memoryless
> [    0.000000] Initializing node 29 as memoryless
> [    0.000000] Initmem setup node 29 as memoryless
> [    0.000000] Initializing node 30 as memoryless
> [    0.000000] Initmem setup node 30 as memoryless
> [    0.000000] Initializing node 31 as memoryless
> [    0.000000] Initmem setup node 31 as memoryless
> [    0.000000] percpu: Embedded 10 pages/cpu s601512 r0 d53848 u655360
> [    0.000000] Fallback order for Node 0: 0 2 
> [    0.000000] Fallback order for Node 1: 1 2 
> [    0.000000] Fallback order for Node 2: 2 
> [    0.000000] Fallback order for Node 3: 3 2 
> [    0.000000] Fallback order for Node 4: 4 2 
> [    0.000000] Fallback order for Node 5: 5 2 
> [    0.000000] Fallback order for Node 6: 6 2 
> [    0.000000] Fallback order for Node 7: 7 2 
> [    0.000000] Fallback order for Node 8: 8 2 
> [    0.000000] Fallback order for Node 9: 9 2 
> [    0.000000] Fallback order for Node 10: 10 2 
> [    0.000000] Fallback order for Node 11: 11 2 
> [    0.000000] Fallback order for Node 12: 12 2 
> [    0.000000] Fallback order for Node 13: 13 2 
> [    0.000000] Fallback order for Node 14: 14 2 
> [    0.000000] Fallback order for Node 15: 15 2 
> [    0.000000] Fallback order for Node 16: 16 2 
> [    0.000000] Fallback order for Node 17: 17 2 
> [    0.000000] Fallback order for Node 18: 18 2 
> [    0.000000] Fallback order for Node 19: 19 2 
> [    0.000000] Fallback order for Node 20: 20 2 
> [    0.000000] Fallback order for Node 21: 21 2 
> [    0.000000] Fallback order for Node 22: 22 2 
> [    0.000000] Fallback order for Node 23: 23 2 
> [    0.000000] Fallback order for Node 24: 24 2 
> [    0.000000] Fallback order for Node 25: 25 2 
> [    0.000000] Fallback order for Node 26: 26 2 
> [    0.000000] Fallback order for Node 27: 27 2 
> [    0.000000] Fallback order for Node 28: 28 2 
> [    0.000000] Fallback order for Node 29: 29 2 
> [    0.000000] Fallback order for Node 30: 30 2 
> [    0.000000] Fallback order for Node 31: 31 2 
> [    0.000000] Built 1 zonelists, mobility grouping on.  Total pages: 4910400
> [    0.000000] Policy zone: Normal
> [    0.000000] Kernel command line: root=/dev/mapper/rhel_ltcden3--lp1-root 
> [    0.000000] random: crng init done
> [    0.000000] Dentry cache hash table entries: 16777216 (order: 11, 134217728 bytes, linear)
> [    0.000000] Inode-cache hash table entries: 8388608 (order: 10, 67108864 bytes, linear)
> [    0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off
> [    0.000000] Memory: 313817216K/314572800K available (15168K kernel code, 5632K rwdata, 17600K rodata, 5440K init, 2721K bss, 755584K reserved, 0K cma-reserved)
> [    0.000000] SLUB: HWalign=128, Order=0-3, MinObjects=0, CPUs=96, Nodes=32
> [    0.000000] ftrace: allocating 38156 entries in 14 pages
> [    0.000000] ftrace: allocated 14 pages with 3 groups
> [    0.000000] trace event string verifier disabled
> [    0.000000] rcu: Hierarchical RCU implementation.
> [    0.000000] rcu: 	RCU restricting CPUs from NR_CPUS=2048 to nr_cpu_ids=96.
> [    0.000000] 	Rude variant of Tasks RCU enabled.
> [    0.000000] 	Tracing variant of Tasks RCU enabled.
> [    0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies.
> [    0.000000] rcu: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=96
> [    0.000000] NR_IRQS: 512, nr_irqs: 512, preallocated irqs: 16
> [    0.000000] xive: Using IRQ range [400000-40005f]
> [    0.000000] xive: Interrupt handling initialized with spapr backend
> [    0.000000] xive: Using priority 7 for all interrupts
> [    0.000000] xive: Using 64kB queues
> [    0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention.
> [    0.000001] time_init: 56 bit decrementer (max: 7fffffffffffff)
> [    0.000021] clocksource: timebase: mask: 0xffffffffffffffff max_cycles: 0x761537d007, max_idle_ns: 440795202126 ns
> [    0.000055] clocksource: timebase mult[1f40000] shift[24] registered
> [    0.000233] Console: colour dummy device 80x25
> [    0.000252] printk: console [hvc0] enabled
> [    0.000252] printk: console [hvc0] enabled
> [    0.000269] printk: bootconsole [udbg0] disabled
> [    0.000269] printk: bootconsole [udbg0] disabled
> [    0.000360] pid_max: default: 98304 minimum: 768
> [    0.000481] LSM: initializing lsm=capability,yama,integrity,selinux,bpf
> [    0.000550] Yama: becoming mindful.
> [    0.000565] SELinux:  Initializing.
> [    0.000568] SELinux: CONFIG_SECURITY_SELINUX_CHECKREQPROT_VALUE is non-zero.  This is deprecated and will be rejected in a future kernel release.
> [    0.000573] SELinux: https://github.com/SELinuxProject/selinux-kernel/wiki/DEPRECATE-checkreqprot
> [    0.000747] LSM support for eBPF active
> [    0.000929] Mount-cache hash table entries: 262144 (order: 5, 2097152 bytes, linear)
> [    0.001056] Mountpoint-cache hash table entries: 262144 (order: 5, 2097152 bytes, linear)
> [    0.003430] cblist_init_generic: Setting adjustable number of callback queues.
> [    0.003454] cblist_init_generic: Setting shift to 7 and lim to 1.
> [    0.003486] cblist_init_generic: Setting shift to 7 and lim to 1.
> [    0.003507] POWER10 performance monitor hardware support registered
> [    0.003537] rcu: Hierarchical SRCU implementation.
> [    0.003540] rcu: 	Max phase no-delay instances is 1000.
> [    0.004445] smp: Bringing up secondary CPUs ...
> [    0.022208] smp: Brought up 1 node, 96 CPUs
> [    0.022218] numa: Node 2 CPUs: 0-95
> [    0.022222] Big cores detected but using small core scheduling
> [    0.033289] devtmpfs: initialized
> [    0.042965] PCI host bridge /pci@800000020000019  ranges:
> [    0.042984]  MEM 0x0000040080000000..0x00000400feffffff -> 0x0000000080000000 
> [    0.042988]  MEM 0x0000044000000000..0x0000047fffffffff -> 0x0006204000000000 
> [    0.043060] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns
> [    0.043066] futex hash table entries: 32768 (order: 6, 4194304 bytes, linear)
> [    0.043643] NET: Registered PF_NETLINK/PF_ROUTE protocol family
> [    0.043801] audit: initializing netlink subsys (disabled)
> [    0.043903] audit: type=2000 audit(1676701683.040:1): state=initialized audit_enabled=0 res=1
> [    0.043954] thermal_sys: Registered thermal governor 'fair_share'
> [    0.043955] thermal_sys: Registered thermal governor 'step_wise'
> [    0.044007] cpuidle: using governor menu
> [    0.044306] pstore: Registered nvram as persistent store backend
> [    0.044682] EEH: pSeries platform initialized
> [    0.055324] PCI: Probing PCI hardware
> [    0.055372] PCI host bridge to bus 0019:01
> [    0.055376] pci_bus 0019:01: root bus resource [mem 0x40080000000-0x400feffffff] (bus address [0x80000000-0xfeffffff])
> [    0.055381] pci_bus 0019:01: root bus resource [mem 0x44000000000-0x47fffffffff 64bit] (bus address [0x6204000000000-0x6207fffffffff])
> [    0.055386] pci_bus 0019:01: root bus resource [bus 01-ff]
> [    0.056001] pci 0019:01:00.0: No hypervisor support for SR-IOV on this device, IOV BARs disabled.
> [    0.064978] IOMMU table initialized, virtual merging enabled
> [    0.065086] pci 0019:01:00.0: Adding to iommu group 0
> [    0.066308] EEH: Capable adapter found: recovery enabled.
> [    0.066550] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible.
> [    0.066812] HugeTLB: registered 2.00 MiB page size, pre-allocated 0 pages
> [    0.066817] HugeTLB: 0 KiB vmemmap can be freed for a 2.00 MiB page
> [    0.066821] HugeTLB: registered 1.00 GiB page size, pre-allocated 0 pages
> [    0.066824] HugeTLB: 0 KiB vmemmap can be freed for a 1.00 GiB page
> [    0.085068] cryptd: max_cpu_qlen set to 1000
> [    0.085368] iommu: Default domain type: Translated 
> [    0.085371] iommu: DMA domain TLB invalidation policy: strict mode 
> [    0.085491] SCSI subsystem initialized
> [    0.085517] usbcore: registered new interface driver usbfs
> [    0.085523] usbcore: registered new interface driver hub
> [    0.085542] usbcore: registered new device driver usb
> [    0.085563] pps_core: LinuxPPS API ver. 1 registered
> [    0.085565] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@xxxxxxxx>
> [    0.085569] PTP clock support registered
> [    0.085628] EDAC MC: Ver: 3.0.0
> [    0.085885] NetLabel: Initializing
> [    0.085887] NetLabel:  domain hash size = 128
> [    0.085889] NetLabel:  protocols = UNLABELED CIPSOv4 CALIPSO
> [    0.085903] NetLabel:  unlabeled traffic allowed by default
> [    0.085956] vgaarb: loaded
> [    0.086157] clocksource: Switched to clocksource timebase
> [    0.086451] VFS: Disk quotas dquot_6.6.0
> [    0.086480] VFS: Dquot-cache hash table entries: 8192 (order 0, 65536 bytes)
> [    0.088080] NET: Registered PF_INET protocol family
> [    0.088237] IP idents hash table entries: 262144 (order: 5, 2097152 bytes, linear)
> [    0.091345] tcp_listen_portaddr_hash hash table entries: 65536 (order: 4, 1048576 bytes, linear)
> [    0.091454] Table-perturb hash table entries: 65536 (order: 2, 262144 bytes, linear)
> [    0.091480] TCP established hash table entries: 524288 (order: 6, 4194304 bytes, linear)
> [    0.092201] TCP bind hash table entries: 65536 (order: 5, 2097152 bytes, linear)
> [    0.092386] TCP: Hash tables configured (established 524288 bind 65536)
> [    0.092708] MPTCP token hash table entries: 65536 (order: 4, 1572864 bytes, linear)
> [    0.092826] UDP hash table entries: 65536 (order: 5, 2097152 bytes, linear)
> [    0.093046] UDP-Lite hash table entries: 65536 (order: 5, 2097152 bytes, linear)
> [    0.093402] NET: Registered PF_UNIX/PF_LOCAL protocol family
> [    0.093413] NET: Registered PF_XDP protocol family
> [    0.093509] PCI: CLS 128 bytes, default 128
> [    0.093638] Unpacking initramfs...
> [    0.094862] vio_register_device_node: node lid missing 'reg'
> [    0.103665] vas: GZIP feature is available
> [    0.104512] hv-24x7: read 548 catalog entries, created 387 event attrs (0 failures), 387 descs
> [    0.108478] Initialise system trusted keyrings
> [    0.108493] Key type blacklist registered
> [    0.108560] workingset: timestamp_bits=38 max_order=23 bucket_order=0
> [    0.108587] zbud: loaded
> [    0.108988] integrity: Platform Keyring initialized
> [    0.119068] NET: Registered PF_ALG protocol family
> [    0.119083] Key type asymmetric registered
> [    0.119086] Asymmetric key parser 'x509' registered
> [    0.119120] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 247)
> [    0.119182] io scheduler mq-deadline registered
> [    0.119185] io scheduler kyber registered
> [    0.119198] io scheduler bfq registered
> [    0.120970] atomic64_test: passed
> [    0.121210] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
> [    0.121214] PowerPC PowerNV PCI Hotplug Driver version: 0.1
> [    0.121532] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
> [    0.121728] Non-volatile memory driver v1.3
> [    0.122174] rdac: device handler registered
> [    0.122215] hp_sw: device handler registered
> [    0.122218] emc: device handler registered
> [    0.122260] alua: device handler registered
> [    0.122450] usbcore: registered new interface driver usbserial_generic
> [    0.122457] usbserial: USB Serial support registered for generic
> [    0.122529] mousedev: PS/2 mouse device common for all mice
> [    0.122644] rtc-generic rtc-generic: registered as rtc0
> [    0.122676] rtc-generic rtc-generic: setting system clock to 2023-02-18T06:28:03 UTC (1676701683)
> [    0.122755] xcede: xcede_record_size = 10
> [    0.122758] xcede: Record 0 : hint = 1, latency = 0x1800 tb ticks, Wake-on-irq = 1
> [    0.122765] xcede: Record 1 : hint = 2, latency = 0x3c00 tb ticks, Wake-on-irq = 0
> [    0.122769] cpuidle: Skipping the 2 Extended CEDE idle states
> [    0.122771] cpuidle: Fixed up CEDE exit latency to 12 us
> [    0.123865] nx_compress_pseries ibm,compression-v1: nx842_OF_upd: max_sync_size new:65536 old:0
> [    0.123871] nx_compress_pseries ibm,compression-v1: nx842_OF_upd: max_sync_sg new:510 old:0
> [    0.123875] nx_compress_pseries ibm,compression-v1: nx842_OF_upd: max_sg_len new:4080 old:0
> [    0.124018] hid: raw HID events driver (C) Jiri Kosina
> [    0.124042] usbcore: registered new interface driver usbhid
> [    0.124048] usbhid: USB HID core driver
> [    0.124069] drop_monitor: Initializing network drop monitor service
> [    0.134168] Initializing XFRM netlink socket
> [    0.134234] NET: Registered PF_INET6 protocol family
> [    0.134747] Segment Routing with IPv6
> [    0.134756] In-situ OAM (IOAM) with IPv6
> [    0.134776] NET: Registered PF_PACKET protocol family
> [    0.134817] mpls_gso: MPLS GSO support
> [    0.134836] secvar-sysfs: secvar: failed to retrieve secvar operations.
> [    0.139134] registered taskstats version 1
> [    0.140964] Loading compiled-in X.509 certificates
> [    0.156881] Loaded X.509 cert 'Build time autogenerated kernel key: 7b04a1cdeda81c05f7a701657946db5548da6201'
> [    0.157494] zswap: loaded using pool lzo/zbud
> [    0.160050] page_owner is disabled
> [    0.160226] pstore: Using crash dump compression: deflate
> [    0.972073] Freeing initrd memory: 76160K
> [    0.974020] Key type encrypted registered
> [    0.974056] Secure boot mode disabled
> [    0.974060] ima: No TPM chip found, activating TPM-bypass!
> [    0.974067] Loading compiled-in module X.509 certificates
> [    0.974567] Loaded X.509 cert 'Build time autogenerated kernel key: 7b04a1cdeda81c05f7a701657946db5548da6201'
> [    0.974571] ima: Allocated hash algorithm: sha256
> [    0.974598] Secure boot mode disabled
> [    0.974615] Trusted boot mode disabled
> [    0.974617] ima: No architecture policies found
> [    0.974629] evm: Initialising EVM extended attributes:
> [    0.974631] evm: security.selinux
> [    0.974633] evm: security.SMACK64 (disabled)
> [    0.974635] evm: security.SMACK64EXEC (disabled)
> [    0.974637] evm: security.SMACK64TRANSMUTE (disabled)
> [    0.974639] evm: security.SMACK64MMAP (disabled)
> [    0.974641] evm: security.apparmor (disabled)
> [    0.974642] evm: security.ima
> [    0.974644] evm: security.capability
> [    0.974646] evm: HMAC attrs: 0x1
> [    0.974676] alg: No test for 842 (842-nx)
> [    0.982519] Freeing unused kernel image (initmem) memory: 5440K
> [    1.086157] Run /init as init process
> [    1.092664] systemd[1]: systemd 239 (239-58.el8) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy)
> [    1.092708] systemd[1]: Detected virtualization powervm.
> [    1.092714] systemd[1]: Detected architecture ppc64-le.
> [    1.092718] systemd[1]: Running in initial RAM disk.
> 
> Welcome to Red Hat Enterprise Linux 8.6 (Ootpa) dracut-049-201.git20220131.el8 (Initramfs)!
> 
> [    1.246394] systemd[1]: Set hostname to <ltcden3-lp1.aus.stglabs.ibm.com>.
> [    1.321102] systemd[1]: Listening on Journal Socket.
> [  OK  ] Listening on Journal Socket.
> [    1.322421] systemd[1]: Started Hardware RNG Entropy Gatherer Daemon.
> [  OK  ] Started Hardware RNG Entropy Gatherer Daemon.
> [    1.323252] systemd[1]: Starting Create list of required static device nodes for the current kernel...
>          Starting Create list of required st…ce nodes for the current kernel...
> [    1.323333] systemd[1]: Reached target Local File Systems.
> [  OK  ] Reached target Local File Systems.
> [    1.324085] systemd[1]: Started Memstrack Anylazing Service.
> [  OK  ] Started Memstrack Anylazing Service.
> [  OK  ] Reached target Slices.
> [  OK  ] Listening on udev Kernel Socket.
> [  OK  ] Listening on Journal Socket (/dev/log).
>          Starting Journal Service...
>          Starting Create Volatile Files and Directories...
> [  OK  ] Reached target Swap.
> [  OK  ] Reached target Timers.
>          Starting Setup Virtual Console...
>          Starting Load Kernel Modules...
> [  OK  ] Listening on udev Control Socket.
> [  OK  ] Reached target Sockets.
> [  OK  ] Started Create list of required sta…vice nodes for the current kernel.
> [  OK  ] Started Create Volatile Files and Directories.
>          Starting Create Static Device Nodes in /dev...
> [FAILED] Failed to start Load Kernel Modules.
> See 'systemctl status systemd-modules-load.service' for details.
>          Starting Apply Kernel Variables...
> [  OK  ] Started Create Static Device Nodes in /dev.
> [  OK  ] Started Apply Kernel Variables.
> [  OK  ] Started Journal Service.
> [  OK  ] Started Setup Virtual Console.
>          Starting dracut cmdline hook...
> [  OK  ] Started dracut cmdline hook.
>          Starting dracut pre-udev hook...
> [    1.477216] device-mapper: core: CONFIG_IMA_DISABLE_HTABLE is disabled. Duplicate IMA measurements will not be recorded in the IMA log.
> [    1.477252] device-mapper: uevent: version 1.0.3
> [    1.477339] device-mapper: ioctl: 4.47.0-ioctl (2022-07-28) initialised: dm-devel@xxxxxxxxxx
> [  OK  ] Started dracut pre-udev hook.
>          Starting udev Kernel Device Manager...
> [  OK  ] Started udev Kernel Device Manager.
>          Starting udev Coldplug all Devices...
>          Mounting Kernel Configuration File System...
> [  OK  ] Mounted Kernel Configuration File System.
> [    1.651355] synth uevent: /devices/vio: failed to send uevent
> [    1.651365] vio vio: uevent: failed to send synthetic uevent: -19
> [    1.651402] synth uevent: /devices/vio/4000: failed to send uevent
> [    1.651406] vio 4000: uevent: failed to send synthetic uevent: -19
> [    1.651415] synth uevent: /devices/vio/4001: failed to send uevent
> [    1.651418] vio 4001: uevent: failed to send synthetic uevent: -19
> [    1.651428] synth uevent: /devices/vio/4002: failed to send uevent
> [    1.651431] vio 4002: uevent: failed to send synthetic uevent: -19
> [    1.651441] synth uevent: /devices/vio/4004: failed to send uevent
> [    1.651444] vio 4004: uevent: failed to send synthetic uevent: -19
> [  OK  ] Started udev Coldplug all Devices.
> [  OK  ] Reached target System Initialization.
>          Starting Show Plymouth Boot Screen...
>          Starting dracut initqueue hook...
> [  OK  ] Started Show Plymouth Boot Screen.
> [  OK  ] Started Forward Password Requests to Plymouth Directory Watch.
> [  OK  ] Reached target Paths.
> [  OK  ] Reached target Basic System.
> [    1.709851] ibmveth 30000002 net0: renamed from eth0
> [    1.712196] nvme nvme0: pci function 0019:01:00.0
> [    1.714161] ibmvscsi 30000065: SRP_VERSION: 16.a
> [    1.714223] ibmvscsi 30000065: Maximum ID: 64 Maximum LUN: 32 Maximum Channel: 3
> [    1.714227] scsi host0: IBM POWER Virtual SCSI Adapter 1.5.9
> [    1.714365] ibmvscsi 30000065: partner initialization complete
> [    1.714391] ibmvscsi 30000065: host srp version: 16.a, host partition ltcden3-vios1 (100), OS 3, max io 524288
> [    1.714421] ibmvscsi 30000065: Client reserve enabled
> [    1.714426] ibmvscsi 30000065: sent SRP login
> [    1.714448] ibmvscsi 30000065: SRP_LOGIN succeeded
> [    1.721611] nvme nvme0: Shutdown timeout set to 8 seconds
> [    1.737117] scsi 0:0:1:0: Direct-Access     AIX      VDASD            0001 PQ: 0 ANSI: 3
> [    1.738038] scsi 0:0:2:0: Direct-Access     AIX      VDASD            0001 PQ: 0 ANSI: 3
> [    1.738364] scsi 0:0:3:0: Direct-Access     AIX      VDASD            0001 PQ: 0 ANSI: 3
> [    1.750754] nvme nvme0: 32/0/0 default/read/poll queues
> [    1.754482]  nvme0n1: p1 p2 p3
> [    1.772868] scsi 0:0:1:0: Attached scsi generic sg0 type 0
> [    1.772898] scsi 0:0:2:0: Attached scsi generic sg1 type 0
> [    1.772920] scsi 0:0:3:0: Attached scsi generic sg2 type 0
> [    1.794722] sd 0:0:2:0: [sda] 629145600 512-byte logical blocks: (322 GB/300 GiB)
> [    1.794757] sd 0:0:2:0: [sda] Write Protect is off
> [    1.794758] sd 0:0:1:0: [sdb] 209715200 512-byte logical blocks: (107 GB/100 GiB)
> [    1.794790] sd 0:0:2:0: [sda] Cache data unavailable
> [    1.794792] sd 0:0:1:0: [sdb] Write Protect is off
> [    1.794796] sd 0:0:2:0: [sda] Assuming drive cache: write through
> [    1.794826] sd 0:0:1:0: [sdb] Cache data unavailable
> [    1.794829] sd 0:0:1:0: [sdb] Assuming drive cache: write through
> [    1.794943] sd 0:0:3:0: [sdc] 91750400 4096-byte logical blocks: (376 GB/350 GiB)
> [    1.794979] sd 0:0:3:0: [sdc] Write Protect is off
> [    1.795008] sd 0:0:3:0: [sdc] Cache data unavailable
> [    1.795011] sd 0:0:3:0: [sdc] Assuming drive cache: write through
> [    1.796625]  sdb: sdb1 sdb2 sdb3
> [    1.796717]  sdc: sdc1
> [    1.796720]  sda: sda1 sda2
> [    1.796752] sd 0:0:1:0: [sdb] Attached SCSI disk
> [    1.796820] sd 0:0:3:0: [sdc] Attached SCSI disk
> [    1.796841] sd 0:0:2:0: [sda] Attached SCSI disk
> [  OK  ] Found device /dev/mapper/rhel_ltcden3--lp1-root.
> [  OK  ] Reached target Initrd Root Device.
> [  OK  ] Started dracut initqueue hook.
>          Starting File System Check on /dev/mapper/rhel_ltcden3--lp1-root...
> [  OK  ] Reached target Remote File Systems (Pre).
> [  OK  ] Reached target Remote File Systems.
> [  OK  ] Started File System Check on /dev/mapper/rhel_ltcden3--lp1-root.
>          Mounting /sysroot...
> [    3.060638] SGI XFS with ACLs, security attributes, quota, no debug enabled
> [    3.062369] XFS (dm-0): Mounting V5 Filesystem 7b801289-75a7-4d39-8cd3-24526e9e9da7
> [    3.109076] XFS (dm-0): Ending clean mount
> [  OK  ] Mounted /sysroot.
> [  OK  ] Reached target Initrd Root File System.
>          Starting Reload Configuration from the Real Root...
> [  OK  ] Started Reload Configuration from the Real Root.
> [  OK  ] Reached target Initrd File Systems.
> [  OK  ] Reached target Initrd Default Target.
>          Starting dracut pre-pivot and cleanup hook...
> [  OK  ] Started dracut pre-pivot and cleanup hook.
>          Starting Cleaning Up and Shutting Down Daemons...
> [  OK  ] Stopped dracut pre-pivot and cleanup hook.
> [  OK  ] Stopped target Remote File Systems.
> [  OK  ] Stopped target Initrd Default Target.
> [  OK  ] Stopped target Timers.
> [  OK  ] Stopped target Initrd Root Device.
> [  OK  ] Stopped target Remote File Systems (Pre).
> [  OK  ] Stopped dracut initqueue hook.
>          Starting Setup Virtual Console...
> [  OK  ] Stopped target Basic System.
> [  OK  ] Stopped target System Initialization.
> [  OK  ] Stopped Create Volatile Files and Directories.
> [  OK  ] Stopped Apply Kernel Variables.
> [  OK  ] Stopped udev Coldplug all Devices.
>          Stopping udev Kernel Device Manager...
> [  OK  ] Stopped target Swap.
> [  OK  ] Stopped target Paths.
> [  OK  ] Stopped target Sockets.
> [  OK  ] Stopped target Slices.
>          Starting Plymouth switch root service...
> [  OK  ] Stopped target Local File Systems.
> [  OK  ] Started Cleaning Up and Shutting Down Daemons.
> [  OK  ] Started Plymouth switch root service.
> [  OK  ] Started Setup Virtual Console.
> [  OK  ] Stopped udev Kernel Device Manager.
> [  OK  ] Stopped Create Static Device Nodes in /dev.
> [  OK  ] Stopped Create list of required sta…vice nodes for the current kernel.
>          Stopping Hardware RNG Entropy Gatherer Daemon...
> [  OK  ] Stopped dracut pre-udev hook.
> [  OK  ] Stopped dracut cmdline hook.
> [  OK  ] Closed udev Control Socket.
> [  OK  ] Closed udev Kernel Socket.
>          Starting Cleanup udevd DB...
> [  OK  ] Stopped Hardware RNG Entropy Gatherer Daemon.
> [  OK  ] Started Cleanup udevd DB.
> [  OK  ] Reached target Switch Root.
>          Starting Switch Root...
> [    3.449370] printk: systemd: 22 output lines suppressed due to ratelimiting
> [    3.593463] SELinux:  Runtime disable is deprecated, use selinux=0 on the kernel cmdline.
> [    3.593471] SELinux:  https://github.com/SELinuxProject/selinux-kernel/wiki/DEPRECATE-runtime-disable
> [   19.036134] SELinux:  Disabled at runtime.
> [   19.196154] audit: type=1404 audit(1676701702.570:2): enforcing=0 old_enforcing=0 auid=4294967295 ses=4294967295 enabled=0 old-enabled=1 lsm=selinux res=1
> [   19.201608] systemd[1]: systemd 239 (239-58.el8) running in system mode. (+PAM +AUDIT +SELINUX +IMA -APPARMOR +SMACK +SYSVINIT +UTMP +LIBCRYPTSETUP +GCRYPT +GNUTLS +ACL +XZ +LZ4 +SECCOMP +BLKID +ELFUTILS +KMOD +IDN2 -IDN +PCRE2 default-hierarchy=legacy)
> [   19.201637] systemd[1]: Detected virtualization powervm.
> [   19.201642] systemd[1]: Detected architecture ppc64-le.
> 
> Welcome to Red Hat Enterprise Linux 8.6 (Ootpa)!
> 
> [   19.202637] systemd[1]: Set hostname to <ltcden3-lp1.aus.stglabs.ibm.com>.
> [   21.777702] systemd[1]: systemd-journald.service: Succeeded.
> [   21.778191] systemd[1]: initrd-switch-root.service: Succeeded.
> [   21.778387] systemd[1]: Stopped Switch Root.
> [  OK  ] Stopped Switch Root.
> [   21.778748] systemd[1]: systemd-journald.service: Service has no hold-off time (RestartSec=0), scheduling restart.
> [   21.778795] systemd[1]: systemd-journald.service: Scheduled restart job, restart counter is at 1.
> [   21.778847] systemd[1]: Stopped Journal Service.
> [  OK  ] Stopped Journal Service.
>          Starting Journal Service...
> [  OK  ] Stopped Plymouth switch root service.
> [  OK  ] Stopped target Switch Root.
> [  OK  ] Listening on udev Kernel Socket.
> [  OK  ] Listening on initctl Compatibility Named Pipe.
> [  OK  ] Listening on Process Core Dump Socket.
>          Mounting Huge Pages File System...
>          Starting Load Kernel Modules...
>          Starting Read and set NIS domainname from /etc/sysconfig/network...
>          Activating swap /dev/mapper/rhel_ltcden3--lp1-swap...
> [  OK  ] Created slice system-serial\x2dgetty.slice.
> [  OK  ] Created slice system-getty.slice.
>          Starting Create list of required st…ce nodes for the current kernel...
> [  OK  ] Created slice User and Session Slice.
> [  OK  ] Reached target Slices.
> [  OK  ] Listening on RPCbind Server Activation Socket.
> [  OK  ] Reached target RPC Port Mapper.
>          Mounting POSIX Message Queue File System...
> [  OK  ] Stopped File System Check on Root Device.
>          Starting Remount Root and Kernel File Systems...
> [  OK  ] Started Forward Password Requests to Wall Directory Watch.
> [  OK  ] Reached target Local Encrypted Volumes.
> [   21.912238] Adding 4194240k swap on /dev/mapper/rhel_ltcden3--lp1-swap.  Priority:-2 extents:1 across:4194240k FS
> [  OK  ] Listening on Device-mapper event daemon FIFOs.
>          Starting Monitoring of LVM2 mirrors…ng dmeventd or progress polling...
>          Starting Setup Virtual Console...
> [  OK  ] Stopped target Initrd Root File System.
>          Mounting Kernel Debug File System...
> [  OK  ] Listening on udev Control Socket.
>          Starting udev Coldplug all Devices...
> [  OK  ] Created slice system-sshd\x2dkeygen.slice.
> [  OK  ] Stopped target Initrd File Systems.
> [  OK  ] Listening on LVM2 poll daemon socket.
> [  OK  ] Activated swap /dev/mapper/rhel_ltcden3--lp1-swap.
> [  OK  ] Mounted Huge Pages File System.
> [  OK  ] Started Read and set NIS domainname from /etc/sysconfig/network.
> [  OK  ] Started Create list of required sta…vice nodes for the current kernel.
> [  OK  ] Started Journal Service.
> [  OK  ] Mounted POSIX Message Queue File System.
> [  OK  ] Mounted Kernel Debug File System.
> [  OK  ] Reached target Swap.
> [   22.217261] xfs filesystem being remounted at / supports timestamps until 2038 (0x7fffffff)
> [  OK  ] Started Remount Root and Kernel File Systems.
> [  OK  ] Started Monitoring of LVM2 mirrors,…sing dmeventd or progress polling.
>          Starting Create Static Device Nodes in /dev...
>          Starting Load/Save Random Seed...
>          Starting Flush Journal to Persistent Storage...
> [FAILED] Failed to start Load Kernel Modules.
> See 'systemctl status systemd-modules-load.service' for details.
>          Starting Apply Kernel Variables...
> [   22.341308] synth uevent: /devices/vio: failed to send uevent
> [   22.341317] vio vio: uevent: failed to send synthetic uevent: -19
> [   22.341597] synth uevent: /devices/vio/4000: failed to send uevent
> [   22.341600] vio 4000: uevent: failed to send synthetic uevent: -19
> [   22.341610] synth uevent: /devices/vio/4001: failed to send uevent
> [   22.341613] vio 4001: uevent: failed to send synthetic uevent: -19
> [   22.341622] synth uevent: /devices/vio/4002: failed to send uevent
> [   22.341625] vio 4002: uevent: failed to send synthetic uevent: -19
> [   22.341633] synth uevent: /devices/vio/4004: failed to send uevent
> [   22.341636] vio 4004: uevent: failed to send synthetic uevent: -19
> [  OK  ] Started udev Coldplug all Devices.
>          Starting udev Wait for Complete Device Initialization...
> [  OK  ] Started Load/Save Random Seed.
> [  OK  ] Started Flush Journal to Persistent Storage.
> [  OK  ] Started Setup Virtual Console.
> [  OK  ] Started Apply Kernel Variables.
> [  OK  ] Started Create Static Device Nodes in /dev.
>          Starting udev Kernel Device Manager...
> [  OK  ] Started udev Kernel Device Manager.
> [   24.100485] pseries_rng: Registering IBM pSeries RNG driver
> [  OK  ] Created slice system-lvm2\x2dpvscan.slice.
>          Starting LVM event activation on device 8:19...
> [  OK  ] Started LVM event activation on device 8:19.
> [  OK  ] Started udev Wait for Complete Device Initialization.
> [  OK  ] Reached target Local File Systems (Pre).
>          Mounting /home...
>          Mounting /home2...
>          Mounting /boot...
> [   25.049055] XFS (nvme0n1p1): Mounting V5 Filesystem 31458cd5-8eb4-48b4-8f5a-112217ef59df
> [   25.049822] XFS (dm-1): Mounting V5 Filesystem 0ac3d742-375d-4cd7-bdd2-72b0559b1957
> [   25.050953] XFS (sdb2): Mounting V5 Filesystem c72725b8-35e3-42b4-b5cb-f8435cada964
> [   25.055813] XFS (nvme0n1p1): Ending clean mount
> [   25.056844] xfs filesystem being mounted at /home supports timestamps until 2038 (0x7fffffff)
> [  OK  ] Mounted /home.
> [   25.389921] XFS (dm-1): Ending clean mount
> [   25.513566] xfs filesystem being mounted at /home2 supports timestamps until 2038 (0x7fffffff)
> [  OK  ] Mounted /home2.
> [   26.797307] XFS (sdb2): Ending clean mount
> [   26.857340] xfs filesystem being mounted at /boot supports timestamps until 2038 (0x7fffffff)
> [  OK  ] Mounted /boot.
> [  OK  ] Reached target Local File Systems.
>          Starting Tell Plymouth To Write Out Runtime Data...
>          Starting Restore /run/initramfs on shutdown...
>          Starting Import network configuration from initramfs...
> [  OK  ] Started Restore /run/initramfs on shutdown.
> [  OK  ] Started Import network configuration from initramfs.
>          Starting Create Volatile Files and Directories...
> [  OK  ] Started Tell Plymouth To Write Out Runtime Data.
> [  OK  ] Started Create Volatile Files and Directories.
>          Mounting RPC Pipe File System...
>          Starting Security Auditing Service...
>          Starting RPC Bind...
> [  OK  ] Started RPC Bind.
> [   27.888702] RPC: Registered named UNIX socket transport module.
> [   27.888714] RPC: Registered udp transport module.
> [   27.888717] RPC: Registered tcp transport module.
> [   27.888719] RPC: Registered tcp NFSv4.1 backchannel transport module.
> [  OK  ] Mounted RPC Pipe File System.
> [  OK  ] Reached target rpc_pipefs.target.
> [  OK  ] Started Security Auditing Service.
>          Starting Update UTMP about System Boot/Shutdown...
> [  OK  ] Started Update UTMP about System Boot/Shutdown.
> [  OK  ] Reached target System Initialization.
> [  OK  ] Listening on Open-iSCSI iscsid Socket.
> [  OK  ] Listening on CUPS Scheduler.
> [  OK  ] Started Run system activity accounting tool every 10 minutes.
> [  OK  ] Started CUPS Scheduler.
> [  OK  ] Reached target Paths.
> [  OK  ] Listening on SSSD Kerberos Cache Manager responder socket.
> [  OK  ] Listening on D-Bus System Message Bus Socket.
> [  OK  ] Started Generate summary of yesterday's process accounting.
> [  OK  ] Started daily update of the root trust anchor for DNSSEC.
> [  OK  ] Started Daily Cleanup of Temporary Directories.
> [  OK  ] Started dnf makecache --timer.
> [  OK  ] Reached target Timers.
> [  OK  ] Listening on Open-iSCSI iscsiuio Socket.
> [  OK  ] Listening on Avahi mDNS/DNS-SD Stack Activation Socket.
> [  OK  ] Reached target Sockets.
> [  OK  ] Reached target Basic System.
>          Starting NTP client/server...
>          Starting Hardware RNG Entropy Gatherer Wake threshold service...
> [  OK  ] Started libstoragemgmt plug-in server daemon.
> [  OK  ] Started irqbalance daemon.
>          Starting Resets System Activity Logs...
>          Starting ppc64-diag rtas_errd (platform error handling) Service...
>          Starting Self Monitoring and Reporting Technology (SMART) Daemon...
> [  OK  ] Reached target sshd-keygen.target.
>          Starting ABRT Automated Bug Reporting Tool...
>          Starting System Security Services Daemon...
>          Starting Authorization Manager...
> [  OK  ] Started D-Bus System Message Bus.
>          Starting Avahi mDNS/DNS-SD Stack...
> [  OK  ] Started Hardware RNG Entropy Gatherer Wake threshold service.
> [  OK  ] Started Hardware RNG Entropy Gatherer Daemon.
> [  OK  ] Started Self Monitoring and Reporting Technology (SMART) Daemon.
> [  OK  ] Started Resets System Activity Logs.
> [  OK  ] Started ppc64-diag rtas_errd (platform error handling) Service.
> [  OK  ] Started Avahi mDNS/DNS-SD Stack.
> [  OK  ] Started NTP client/server.
> [  OK  ] Started System Security Services Daemon.
> [  OK  ] Reached target User and Group Name Lookups.
>          Starting Login Service...
> [  OK  ] Started Login Service.
> [  OK  ] Started Authorization Manager.
>          Starting firewalld - dynamic firewall daemon...
>          Starting Modem Manager...
> [  OK  ] Started Modem Manager.
> [  OK  ] Started firewalld - dynamic firewall daemon.
> [  OK  ] Reached target Network (Pre).
>          Starting Network Manager...
> [  OK  ] Started Network Manager.
>          Starting hybrid virtual network scan and config...
>          Starting Network Manager Wait Online...
> [  OK  ] Reached target Network.
>          Starting Enable periodic update of entitlement certificates....
>          Starting CUPS Scheduler...
>          Starting GSSAPI Proxy Daemon...
>          Starting OpenSSH server daemon...
>          Starting Logout off all iSCSI sessions on shutdown...
> [  OK  ] Started Logout off all iSCSI sessions on shutdown.
> [  OK  ] Started Enable periodic update of entitlement certificates..
>          Starting Hostname Service...
> [  OK  ] Started ABRT Automated Bug Reporting Tool.
> [  OK  ] Started ABRT kernel log watcher.
> [  OK  ] Started ABRT Xorg log watcher.
> [  OK  ] Started Creates ABRT problems from coredumpctl messages.
> [  OK  ] Started Hostname Service.
> [  OK  ] Listening on Load/Save RF Kill Switch Status /dev/rfkill Watch.
> [FAILED] Failed to start hybrid virtual network scan and config.
> See 'systemctl status hcn-init.service' for details.
> [  OK  ] Started OpenSSH server daemon.
> [  OK  ] Started GSSAPI Proxy Daemon.
> [  OK  ] Reached target NFS client services.
>          Starting Network Manager Script Dispatcher Service...
> [  OK  ] Started CUPS Scheduler.
> [  OK  ] Started Network Manager Script Dispatcher Service.
> [  OK  ] Started Network Manager Wait Online.
> [  OK  ] Reached target Network is Online.
>          Starting Notify NFS peers of a restart...
> [  OK  ] Reached target Remote File Systems (Pre).
> [  OK  ] Reached target Remote File Systems.
>          Starting Harvest vmcores for ABRT...
>          Starting Permit User Sessions...
>          Starting System Logging Service...
> [  OK  ] Started Notify NFS peers of a restart.
> [  OK  ] Started Permit User Sessions.
> [  OK  ] Started Job spooling tools.
>          Starting Terminate Plymouth Boot Screen...
> [  OK  ] Started Command Scheduler.
>          Starting Hold until boot process finishes up...
> [  OK  ] Started System Logging Service.
> [   44.801795] ------------[ cut here ]------------
> [   44.801810] WARNING: CPU: 24 PID: 1906 at fs/xfs/xfs_inode.c:1839 xfs_iunlink_lookup+0x58/0x80 [xfs]
> [   44.801895] Modules linked in: nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set rfkill nf_tables nfnetlink sunrpc pseries_rng xts vmx_crypto xfs libcrc32c sd_mod sg ibmvscsi nvme ibmveth scsi_transport_srp nvme_core t10_pi crc64_rocksoft crc64 dm_mirror dm_region_hash dm_log dm_mod
> [   44.801926] CPU: 24 PID: 1906 Comm: in:imjournal Not tainted 6.2.0-rc8ssh #35
> [   44.801930] Hardware name: IBM,9080-HEX POWER10 (raw) 0x800200 0xf000006 of:IBM,FW1010.22 (NH1010_122) hv:phyp pSeries
> [   44.801933] NIP:  c00800000062fa80 LR: c00800000062fa4c CTR: c000000000ea4d40
> [   44.801936] REGS: c00000001984b690 TRAP: 0700   Not tainted  (6.2.0-rc8ssh)
> [   44.801939] MSR:  8000000000029033 <SF,EE,ME,IR,DR,RI,LE>  CR: 24224842  XER: 00000000
> [   44.801947] CFAR: c00800000062fa54 IRQMASK: 0 
> [   44.801947] GPR00: c0000000091cbec8 c00000001984b930 c0080000006d8300 0000000000000000 
> [   44.801947] GPR04: 00000000002ec44d 0000000000000000 0000000000000000 c000000051aa4d90 
> [   44.801947] GPR08: 0000000000000000 c000000051aa4e40 0000000000000000 fffffffffffffffd 
> [   44.801947] GPR12: 0000000000000040 c000004afece5880 0000000000000000 0000000004000000 
> [   44.801947] GPR16: c00000001984bb38 c00000001984ba38 c00000001984ba68 c00000002c323c00 
> [   44.801947] GPR20: c0000000091cbe00 0000000000008000 c0000000516e5800 00000000002ec44d 
> [   44.801947] GPR24: 000000000030af4d 000000000000000d c000000008d71680 00000000002ec44d 
> [   44.801947] GPR28: c00000002c322400 c0000000091cbe00 c000000054c05618 000000000030af4d 
> [   44.801983] NIP [c00800000062fa80] xfs_iunlink_lookup+0x58/0x80 [xfs]
> [   44.802037] LR [c00800000062fa4c] xfs_iunlink_lookup+0x24/0x80 [xfs]
> [   44.802090] Call Trace:
> [   44.802091] [c00000001984b930] [c000000054c05618] 0xc000000054c05618 (unreliable)
> [   44.802096] [c00000001984b950] [c008000000630094] xfs_iunlink+0x1bc/0x280 [xfs]
> [   44.802149] [c00000001984b9d0] [c008000000634754] xfs_rename+0x69c/0xd10 [xfs]
> [   44.802202] [c00000001984bb10] [c00800000062e020] xfs_vn_rename+0xf8/0x1f0 [xfs]
> [   44.802255] [c00000001984bba0] [c000000000579efc] vfs_rename+0x9bc/0xdf0
> [   44.802261] [c00000001984bc90] [c00000000058018c] do_renameat2+0x3dc/0x5c0
> [   44.802265] [c00000001984bde0] [c000000000580520] sys_rename+0x60/0x80
> [   44.802269] [c00000001984be10] [c000000000033630] system_call_exception+0x150/0x3b0
> [   44.802274] [c00000001984be50] [c00000000000c554] system_call_common+0xf4/0x258
> [   44.802280] --- interrupt: c00 at 0x7fffa6482e20
> [   44.802282] NIP:  00007fffa6482e20 LR: 00007fffa6055e24 CTR: 0000000000000000
> [   44.802285] REGS: c00000001984be80 TRAP: 0c00   Not tainted  (6.2.0-rc8ssh)
> [   44.802288] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 2a082202  XER: 00000000
> [   44.802297] IRQMASK: 0 
> [   44.802297] GPR00: 0000000000000026 00007fffa536d220 00007fffa6607300 00007fffa536d288 
> [   44.802297] GPR04: 000000014f7a6b70 0000000000000000 0700000000000000 0000000000000002 
> [   44.802297] GPR08: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
> [   44.802297] GPR12: 0000000000000000 00007fffa53766e0 000000014f795930 0000000000000000 
> [   44.802297] GPR16: 0000000126b67648 00007fffa6057ce0 0000000000000003 000000000000002b 
> [   44.802297] GPR20: 0000000000000212 00007fffa536e3a0 00007fffa536e398 00007fff98008690 
> [   44.802297] GPR24: 0000000000000000 00007fffa536e388 00007fffa536e378 00007fffa536e3b0 
> [   44.802297] GPR28: 00007fffa6070000 00007fffa60705e0 00007fff98000c10 00007fffa536d288 
> [   44.802331] NIP [00007fffa6482e20] 0x7fffa6482e20
> [   44.802334] LR [00007fffa6055e24] 0x7fffa6055e24
> [   44.802336] --- interrupt: c00
> [   44.802338] Code: 2c230000 4182002c e9230020 2fa90000 419e0020 38210020 e8010010 7c0803a6 4e800020 60000000 60000000 60000000 <0fe00000> 60000000 60000000 60000000 
> [   44.802352] ---[ end trace 0000000000000000 ]---
> [   44.802356] XFS (dm-0): Internal error xfs_trans_cancel at line 1097 of file fs/xfs/xfs_trans.c.  Caller xfs_rename+0x9cc/0xd10 [xfs]
> [   44.802412] CPU: 24 PID: 1906 Comm: in:imjournal Tainted: G        W          6.2.0-rc8ssh #35
> [   44.802415] Hardware name: IBM,9080-HEX POWER10 (raw) 0x800200 0xf000006 of:IBM,FW1010.22 (NH1010_122) hv:phyp pSeries
> [   44.802418] Call Trace:
> [   44.802420] [c00000001984b8f0] [c000000000e87328] dump_stack_lvl+0x6c/0x9c (unreliable)
> [   44.802425] [c00000001984b920] [c008000000616a84] xfs_error_report+0x5c/0x80 [xfs]
> [   44.802478] [c00000001984b980] [c0080000006467b0] xfs_trans_cancel+0x178/0x1b0 [xfs]
> [   44.802533] [c00000001984b9d0] [c008000000634a84] xfs_rename+0x9cc/0xd10 [xfs]
> [   44.802586] [c00000001984bb10] [c00800000062e020] xfs_vn_rename+0xf8/0x1f0 [xfs]
> [   44.802641] [c00000001984bba0] [c000000000579efc] vfs_rename+0x9bc/0xdf0
> [   44.802644] [c00000001984bc90] [c00000000058018c] do_renameat2+0x3dc/0x5c0
> [   44.802648] [c00000001984bde0] [c000000000580520] sys_rename+0x60/0x80
> [   44.802652] [c00000001984be10] [c000000000033630] system_call_exception+0x150/0x3b0
> [   44.802657] [c00000001984be50] [c00000000000c554] system_call_common+0xf4/0x258
> [   44.802661] --- interrupt: c00 at 0x7fffa6482e20
> [   44.802664] NIP:  00007fffa6482e20 LR: 00007fffa6055e24 CTR: 0000000000000000
> [   44.802667] REGS: c00000001984be80 TRAP: 0c00   Tainted: G        W           (6.2.0-rc8ssh)
> [   44.802670] MSR:  800000000280f033 <SF,VEC,VSX,EE,PR,FP,ME,IR,DR,RI,LE>  CR: 2a082202  XER: 00000000
> [   44.802678] IRQMASK: 0 
> [   44.802678] GPR00: 0000000000000026 00007fffa536d220 00007fffa6607300 00007fffa536d288 
> [   44.802678] GPR04: 000000014f7a6b70 0000000000000000 0700000000000000 0000000000000002 
> [   44.802678] GPR08: 0000000000000000 0000000000000000 0000000000000000 0000000000000000 
> [   44.802678] GPR12: 0000000000000000 00007fffa53766e0 000000014f795930 0000000000000000 
> [   44.802678] GPR16: 0000000126b67648 00007fffa6057ce0 0000000000000003 000000000000002b 
> [   44.802678] GPR20: 0000000000000212 00007fffa536e3a0 00007fffa536e398 00007fff98008690 
> [   44.802678] GPR24: 0000000000000000 00007fffa536e388 00007fffa536e378 00007fffa536e3b0 
> [   44.802678] GPR28: 00007fffa6070000 00007fffa60705e0 00007fff98000c10 00007fffa536d288 
> [   44.802712] NIP [00007fffa6482e20] 0x7fffa6482e20
> [   44.802715] LR [00007fffa6055e24] 0x7fffa6055e24
> [   44.802717] --- interrupt: c00
> [   44.804354] XFS (dm-0): Corruption of in-memory data (0x8) detected at xfs_trans_cancel+0x190/0x1b0 [xfs] (fs/xfs/xfs_trans.c:1098).  Shutting down filesystem.
> [   44.804413] XFS (dm-0): Please unmount the filesystem and rectify the problem(s)
> 
> 
> 
> >> [  333.390615] Modules linked in: nft_fib_inet nft_fib_ipv4 nft_fib_ipv6 nft_fib nft_reject_inet nf_reject_ipv4 nf_reject_ipv6 nft_reject nft_ct nft_chain_nat nf_nat nf_conntrack nf_defrag_ipv6 nf_defrag_ipv4 ip_set nf_tables nfnetlink rfkill sunrpc pseries_rng xts vmx_crypto xfs libcrc32c sd_mod sg ibmvscsi ibmveth scsi_transport_srp nvme nvme_core t10_pi crc64_rocksoft crc64 dm_mirror dm_region_hash dm_log dm_mod
> >> [  333.390645] CPU: 56 PID: 12450 Comm: rm Not tainted 6.2.0-rc4ssh+ #4
> >> [  333.390649] Hardware name: IBM,9080-HEX POWER10 (raw) 0x800200 0xf000006 of:IBM,FW1010.22 (NH1010_122) hv:phyp pSeries
> >> [  333.390652] NIP:  c0080000004bfa80 LR: c0080000004bfa4c CTR: c000000000ea28d0
> >> [  333.390655] REGS: c0000000442bb8c0 TRAP: 0700   Not tainted  (6.2.0-rc4ssh+)
> >> [  333.390658] MSR:  8000000000029033 <SF,EE,ME,IR,DR,RI,LE>  CR: 24002842  XER: 00000000
> >> [  333.390666] CFAR: c0080000004bfa54 IRQMASK: 0
> >> [  333.390666] GPR00: c00000003b69c0c8 c0000000442bbb60 c008000000568300 0000000000000000
> >> [  333.390666] GPR04: 00000000002ec44d 0000000000000000 0000000000000000 c000000004b27d78
> >> [  333.390666] GPR08: 0000000000000000 c000000004b27e28 0000000000000000 fffffffffffffffd
> >> [  333.390666] GPR12: 0000000000000040 c000004afecc5880 0000000106620918 0000000000000001
> >> [  333.390666] GPR16: 000000010bd36e10 0000000106620dc8 0000000106620e58 0000000106620e90
> >> [  333.390666] GPR20: 0000000106620e30 c0000000880ba938 0000000000200000 00000000002ec44d
> >> [  333.390666] GPR24: 000000000008170d 000000000000000d c0000000519f4800 00000000002ec44d
> >> [  333.390666] GPR28: c0000000880ba800 c00000003b69c000 c0000000833edd20 000000000008170d
> >> [  333.390702] NIP [c0080000004bfa80] xfs_iunlink_lookup+0x58/0x80 [xfs]
> >> [  333.390756] LR [c0080000004bfa4c] xfs_iunlink_lookup+0x24/0x80 [xfs]
> >> [  333.390810] Call Trace:
> >> [  333.390811] [c0000000442bbb60] [c0000000833edd20] 0xc0000000833edd20 (unreliable)
> >> [  333.390816] [c0000000442bbb80] [c0080000004c0094] xfs_iunlink+0x1bc/0x280 [xfs]
> >> [  333.390869] [c0000000442bbc00] [c0080000004c3f84] xfs_remove+0x1dc/0x310 [xfs]
> >> [  333.390922] [c0000000442bbc70] [c0080000004be180] xfs_vn_unlink+0x68/0xf0 [xfs]
> >> [  333.390975] [c0000000442bbcd0] [c000000000576b24] vfs_unlink+0x1b4/0x3d0
> > 
> > ...that trips when rm tries to remove a file, which means that the call
> > stack is
> > 
> > xfs_remove -> xfs_iunlink -> xfs_iunlink_insert_inode ->
> > xfs_iunlink_update_backref -> xfs_iunlink_lookup <kaboom>
> > 
> > It looks as though "rm foo" unlinked foo from the directory and was
> > trying to insert it at the head of one of the unlinked lists in the AGI
> > buffer.  The AGI claims that the list points to an ondisk inode, so the
> > iunlink code tries to find the incore inode to update the incore list,
> > fails to find an incore inode, and this is the result...
> 
> This seems to happen for rename as well. i.e xfs_rename. 
> Does  rename path calls rm first, and then create?

Effectively, yes.  A "mv a b" will unlink b, and that calls the same
internal unlink code as an unlink syscall.

> >>
> >>
> >> we did a git bisect between 5.17 and 6.0. Bisect points to commit 04755d2e5821 
> >> as the bad commit.
> >> Short description of commit:
> >> commit 04755d2e5821b3afbaadd09fe5df58d04de36484 (refs/bisect/bad)
> >> Author: Dave Chinner <dchinner@xxxxxxxxxx>
> >> Date:   Thu Jul 14 11:42:39 2022 +1000
> >>
> >>     xfs: refactor xlog_recover_process_iunlinks()
> > 
> > ...which was in the middle of the series that reworked thev mount time
> > iunlink clearing.  Oddly, I don't spot any obvious errors in /that/
> > patch that didn't already exist.  But this does make me wonder, does
> > xfs_repair -n have anything to say about unlinked or orphaned inodes?
> > 
> > The runtime code expects that every ondisk inode in an iunlink chain has
> > an incore inode that is linked (via i_{next,prev}_unlinked) to the other
> > incore inodes in that same chain.  If this requirement is not met, then
> > the WARNings you see will trip, and the fs shuts down.
> > 
> > My hypothesis here is that one of the AGs has an unprocessed unlinked
> > list.  At mount time, the ondisk log was clean, so mount time log
> > recovery didn't invoke xlog_recover_process_iunlinks, and the list was
> > not cleared.  The mount code does not construct the incore unlinked list
> > from an existing ondisk iunlink list, hence the WARNing.  Prior to 5.17,
> > we only manipulated the ondisk unlink list, and the code never noticed
> > or cared if there were mystery inodes in the list that never went away.
> > 
> > (Obviously, if something blew up earlier in dmesg, that would be
> > relevant here.)
> > 
> > It's possible that we could end up in this situation (clean log,
> > unlinked inodes) if a previous log recovery was only partially
> > successful at clearing the unlinked list, since all that code ignores
> > errors.  If that happens, we ... succeed at mounting and clean the log.
> > 
> > If you're willing to patch your kernels, it would be interesting
> > to printk if the xfs_read_agi or the xlog_recover_iunlink_bucket calls
> > in xlog_recover_iunlink_ag returns an error code.  It might be too late
> 
> We can try. Please provide the Patch. 
> 
> > to capture that, hence my suggestion of seeing if xfs_repair -n will
> > tell us anything else.
> > 
> 
> Could you please clarify? We should run xfs_repair -n from 5.17-rc2 kernel? 

Whatever xfs_repair is installed on the system should suffice to report
an unlinked inode list and any other errors on the filesystem.  That
evidence will guide us towards a kernel patch.

--D

> 
> > I've long thought that the iunlink recovery ought to complain loudly and
> > fail the mount if it can't clear all the unlinked files.  Given the new
> > iunlink design, I think it's pretty much required now.  The uglier piece
> > is that now we either (a) have to clear iunlinks at mount time
> > unconditionally as Eric has been saying for years; or (b) construct the
> > incore list at a convenient time so that the incore list always exists.
> > 
> > Thanks for the detailed report!
> > 
> > --D
> > 
> >>
> >> Git bisect log:
> >> git bisect start
> >> # good: [26291c54e111ff6ba87a164d85d4a4e134b7315c] Linux 5.17-rc2
> >> git bisect good 26291c54e111ff6ba87a164d85d4a4e134b7315c
> >> # bad: [4fe89d07dcc2804c8b562f6c7896a45643d34b2f] Linux 6.0
> >> git bisect bad 4fe89d07dcc2804c8b562f6c7896a45643d34b2f
> >> # good: [d7227785e384d4422b3ca189aa5bf19f462337cc] Merge tag 'sound-5.19-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/tiwai/sound
> >> git bisect good d7227785e384d4422b3ca189aa5bf19f462337cc
> >> # good: [526942b8134cc34d25d27f95dfff98b8ce2f6fcd] Merge tag 'ata-5.20-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/dlemoal/libata
> >> git bisect good 526942b8134cc34d25d27f95dfff98b8ce2f6fcd
> >> # good: [328141e51e6fc79d21168bfd4e356dddc2ec7491] Merge tag 'mmc-v5.20' of git://git.kernel.org/pub/scm/linux/kernel/git/ulfh/mmc
> >> git bisect good 328141e51e6fc79d21168bfd4e356dddc2ec7491
> >> # bad: [eb555cb5b794f4e12a9897f3d46d5a72104cd4a7] Merge tag '5.20-rc-ksmbd-server-fixes' of git://git.samba.org/ksmbd
> >> git bisect bad eb555cb5b794f4e12a9897f3d46d5a72104cd4a7
> >> # bad: [f20c95b46b8fa3ad34b3ea2e134337f88591468b] Merge tag 'tpmdd-next-v5.20' of git://git.kernel.org/pub/scm/linux/kernel/git/jarkko/linux-tpmdd
> >> git bisect bad f20c95b46b8fa3ad34b3ea2e134337f88591468b
> >> # bad: [fad235ed4338749a66ddf32971d4042b9ef47f44] Merge tag 'arm-late-6.0' of git://git.kernel.org/pub/scm/linux/kernel/git/soc/soc
> >> git bisect bad fad235ed4338749a66ddf32971d4042b9ef47f44
> >> # good: [e495274793ea602415d050452088a496abcd9e6c] Merge tag 'for-linus' of git://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma
> >> git bisect good e495274793ea602415d050452088a496abcd9e6c
> >> # good: [9daee913dc8d15eb65e0ff560803ab1c28bb480b] Merge tag 'ext4_for_linus' of git://git.kernel.org/pub/scm/linux/kernel/git/tytso/ext4
> >> git bisect good 9daee913dc8d15eb65e0ff560803ab1c28bb480b
> >> # bad: [29b1d469f3f6842ee4115f0b21f018fc44176468] Merge tag 'trace-rtla-v5.20' of git://git.kernel.org/pub/scm/linux/kernel/git/rostedt/linux-trace
> >> git bisect bad 29b1d469f3f6842ee4115f0b21f018fc44176468
> >> # good: [932b42c66cb5d0ca9800b128415b4ad6b1952b3e] xfs: replace XFS_IFORK_Q with a proper predicate function
> >> git bisect good 932b42c66cb5d0ca9800b128415b4ad6b1952b3e
> >> # bad: [35c5a09f5346e690df7ff2c9075853e340ee10b3] Merge tag 'xfs-buf-lockless-lookup-5.20' of git://git.kernel.org/pub/scm/linux/kernel/git/dgc/linux-xfs into xfs-5.20-mergeB
> >> git bisect bad 35c5a09f5346e690df7ff2c9075853e340ee10b3
> >> # bad: [fad743d7cd8bd92d03c09e71f29eace860f50415] xfs: add log item precommit operation
> >> git bisect bad fad743d7cd8bd92d03c09e71f29eace860f50415
> >> # bad: [04755d2e5821b3afbaadd09fe5df58d04de36484] xfs: refactor xlog_recover_process_iunlinks()
> >> git bisect bad 04755d2e5821b3afbaadd09fe5df58d04de36484
> >> # good: [a4454cd69c66bf3e3bbda352b049732f836fc6b2] xfs: factor the xfs_iunlink functions
> >> git bisect good a4454cd69c66bf3e3bbda352b049732f836fc6b2
> >> Bisecting: 0 revisions left to test after this (roughly 0 steps)
> >> [4fcc94d653270fcc7800dbaf3b11f78cb462b293] xfs: track the iunlink list pointer in the xfs_inode
> >>
> >>
> >> Please reach out, in case any more details are needed. sent with very limited
> >> knowledge of xfs system. these logs are from 5.19 kernel.
> >>
> >> # xfs_info /home
> >> meta-data=/dev/nvme0n1p1         isize=512    agcount=4, agsize=13107200 blks
> >>          =                       sectsz=4096  attr=2, projid32bit=1
> >>          =                       crc=1        finobt=1, sparse=1, rmapbt=0
> >>          =                       reflink=1    bigtime=0 inobtcount=0
> >> data     =                       bsize=4096   blocks=52428800, imaxpct=25
> >>          =                       sunit=0      swidth=0 blks
> >> naming   =version 2              bsize=4096   ascii-ci=0, ftype=1
> >> log      =internal log           bsize=4096   blocks=25600, version=2
> >>          =                       sectsz=4096  sunit=1 blks, lazy-count=1
> >> realtime =none                   extsz=4096   blocks=0, rtextents=0
> >>
> >> # xfs_info -V
> >> xfs_info version 5.0.0
> >>
> >> # uname -a
> >> 5.19.0-rc2



[Index of Archives]     [XFS Filesystem Development (older mail)]     [Linux Filesystem Development]     [Linux Audio Users]     [Yosemite Trails]     [Linux Kernel]     [Linux RAID]     [Linux SCSI]


  Powered by Linux