David, This is related to the lustre integration that I've done. It is necessary to summarize results as expediently as possible, which we hope justify continuation of this work. To that end, although I had a lot of stability problems with the very early fscache code in EL5 kernels, I was able to come up with a testable configuration that demonstrates that reading from fscache is faster in appropriate circumstances. However, I was using a cache disk subsystem that can be read by (iozone / ext3) at 1GB/s, but the (iozone / lustre / fscache /ext3 / fast disk) read performance was just under half that. In the old fscache code, (iozone / Lustre / fscache / ext3 / fast disk) performance = 0.45 (iozone / ext3 / fast disk). Bringing Lustre up to modern kernels is a separate piece of work, and it's just not ready yet. For analysis purposes, we decided to test the (iozone / nfs / fscache /ext3 / fast disk) performance on the very latest fscache code - if the latest code incurs less performance degradation, that would support the assertion that when Lustre is moved to late-model kernels (and therefore late model fscache code) we'll get a [hopefully large] performance gain. First, do you have any performance data that might shed light on this? I would be a bit surprised if you've tested with disk that's this fast... Secondly, I've built the GIT kernel with your nfs+fscache-42 patches applied (all went in cleanly). I'm running this kernel on an otherwise clean Fedora 10 system. fscache and cachefiles modules load cleanly, but I can't get cachefilesd to run. If I "yum install cachefilesd", I get version 0.7-5, and it complains that /proc/fs/cachefiles is missing. If I build cachefilesd-0.9, I a kernel BUG. The relevant lines from /var/log/messages are attached. Am I using the wrong version of cachefilesd? Is it clear what I'm doing wrong? Thanks, John Groves
Jan 2 10:16:38 localhost cachefilesd[4042]: Unable to open /proc/fs/cachefiles: errno 2 (No such file or directory) Jan 2 10:16:50 localhost kernel: Slow work thread pool: Starting up Jan 2 10:16:50 localhost kernel: Slow work thread pool: Ready Jan 2 10:16:50 localhost kernel: FS-Cache: Loaded Jan 2 10:16:50 localhost kernel: CacheFiles: Loaded Jan 2 10:16:56 localhost cachefilesd[4052]: Unable to open /proc/fs/cachefiles: errno 2 (No such file or directory) Jan 2 10:41:27 localhost cachefilesd[4133]: About to bind cache Jan 2 10:41:27 localhost kernel: BUG: unable to handle kernel NULL pointer dereference at 0000000000000000 Jan 2 10:41:27 localhost kernel: IP: [<ffffffff8105dd51>] prepare_kernel_cred+0x4e/0xb9 Jan 2 10:41:27 localhost kernel: PGD 11fc38067 PUD 11fc21067 PMD 0 Jan 2 10:41:27 localhost kernel: Oops: 0002 [#1] SMP Jan 2 10:41:27 localhost kernel: last sysfs file: /sys/devices/pci0000:00/0000:00:09.0/irq Jan 2 10:41:27 localhost kernel: Dumping ftrace buffer: Jan 2 10:41:27 localhost kernel: (ftrace buffer empty) Jan 2 10:41:27 localhost kernel: CPU 1 Jan 2 10:41:27 localhost kernel: Modules linked in: cachefiles fscache fuse nfs lockd nfs_acl sco bridge stp bnep l2cap bluetooth sunrpc ipv6 dm_multipath uinput ib_mthca jedec_probe cfi_probe gen_probe firewire_ohci usb_storage cfi_util ib_mad firewire_core ppdev ib_core mtd k8temp shpchp i2c_nforce2 parport_pc hwmon forcedeth crc_itu_t pcspkr chipreg serio_raw parport i2c_core joydev pata_amd map_funcs ata_generic pata_acpi sata_nv [last unloaded: freq_table] Jan 2 10:41:27 localhost kernel: Pid: 4133, comm: cachefilesd Tainted: G W 2.6.28-rc5 #1 Jan 2 10:41:27 localhost kernel: RIP: 0010:[<ffffffff8105dd51>] [<ffffffff8105dd51>] prepare_kernel_cred+0x4e/0xb9 Jan 2 10:41:27 localhost kernel: RSP: 0018:ffff880122c29d08 EFLAGS: 00010202 Jan 2 10:41:27 localhost kernel: RAX: 0000000000000000 RBX: ffff880069dae600 RCX: 0000000000000001 Jan 2 10:41:27 localhost kernel: RDX: ffff88002802b1d0 RSI: 00000000000000d0 RDI: ffff880121d8d240 Jan 2 10:41:27 localhost kernel: RBP: ffff880122c29d18 R08: ffff88011fc546a8 R09: 0000000000000000 Jan 2 10:41:27 localhost kernel: R10: 00007fffed44c610 R11: 000000c022c29eca R12: ffff880121d8d240 Jan 2 10:41:27 localhost kernel: R13: ffff88007e94c400 R14: ffff88007f1b54b0 R15: 0000000000000004 Jan 2 10:41:27 localhost kernel: FS: 00007f09e54366f0(0000) GS:ffff88007fc01e80(0000) knlGS:0000000000000000 Jan 2 10:41:27 localhost kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b Jan 2 10:41:27 localhost kernel: CR2: 0000000000000000 CR3: 0000000122cad000 CR4: 00000000000006e0 Jan 2 10:41:27 localhost kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 Jan 2 10:41:27 localhost kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400 Jan 2 10:41:27 localhost kernel: Process cachefilesd (pid: 4133, threadinfo ffff880122c28000, task ffff880122c50000) Jan 2 10:41:27 localhost kernel: Stack: Jan 2 10:41:27 localhost kernel: ffff88007e94c400 00000000fffffff4 ffff880122c29d48 ffffffffa02ab3a5 Jan 2 10:41:27 localhost kernel: ffff880122c50000 ffff88007e94c400 ffff88007f1b54b4 ffff88007e94c400 Jan 2 10:41:27 localhost kernel: ffff880122c29e98 ffffffffa02a7098 ffff880122c29e08 ffffffff810d182d Jan 2 10:41:27 localhost kernel: Call Trace: Jan 2 10:41:27 localhost kernel: [<ffffffffa02ab3a5>] cachefiles_get_security_ID+0x24/0x8b [cachefiles] Jan 2 10:41:27 localhost kernel: [<ffffffffa02a7098>] cachefiles_daemon_add_cache+0x1a/0x5a6 [cachefiles] Jan 2 10:41:27 localhost kernel: [<ffffffff810d182d>] ? kern_path+0x25/0x4c Jan 2 10:41:27 localhost kernel: [<ffffffff8102977b>] ? default_spin_lock_flags+0x9/0xe Jan 2 10:41:27 localhost kernel: [<ffffffff810c0df4>] ? virt_to_head_page+0x31/0x41 Jan 2 10:41:27 localhost kernel: [<ffffffff812ae6f6>] ? sock_destroy_inode+0x14/0x16 Jan 2 10:41:27 localhost kernel: [<ffffffff810c0df4>] ? virt_to_head_page+0x31/0x41 Jan 2 10:41:27 localhost kernel: [<ffffffff810d6208>] ? __d_free+0x4b/0x50 Jan 2 10:41:27 localhost kernel: [<ffffffff810d623f>] ? d_free+0x32/0x48 Jan 2 10:41:27 localhost kernel: [<ffffffffa02a7725>] cachefiles_daemon_bind+0x101/0x128 [cachefiles] Jan 2 10:41:27 localhost kernel: [<ffffffffa02a7b15>] cachefiles_daemon_write+0x1a5/0x1cc [cachefiles] Jan 2 10:41:27 localhost kernel: [<ffffffff810c871f>] vfs_write+0xab/0x105 Jan 2 10:41:27 localhost kernel: [<ffffffff810c883d>] sys_write+0x47/0x6f Jan 2 10:41:27 localhost kernel: [<ffffffff810110fa>] system_call_fastpath+0x16/0x1b Jan 2 10:41:27 localhost kernel: Code: 8a 00 00 00 4d 85 e4 74 10 49 8b bc 24 f8 03 00 00 f0 ff 07 49 89 fc eb 0e f0 ff 05 8a 0b 4a 00 49 c7 c4 d0 e8 4f 81 48 8b 43 70 <f0> ff 00 48 8b 43 78 f0 ff 00 f0 ff 05 fe 0b 4a 00 ba d0 00 00 Jan 2 10:41:27 localhost kernel: RIP [<ffffffff8105dd51>] prepare_kernel_cred+0x4e/0xb9 Jan 2 10:41:27 localhost kernel: RSP <ffff880122c29d08> Jan 2 10:41:27 localhost kernel: CR2: 0000000000000000 Jan 2 10:41:27 localhost kernel: ---[ end trace 2172a35469adf1dc ]---
-- Linux-cachefs mailing list Linux-cachefs@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/linux-cachefs
