Ext4 speedup by storing metadata and data on separate devices

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hello all,

This suggestion is not about storing the journal on a separate device.

Many of the tasks on an Ext4 file-system require a full or massive scan of the metadata. A few examples: - backup: you need to get a list with all "mtime" or "size" changed files since last backup - reporting: you need to get a list with all files of a particular "group" owner ID
- delete: deleting the "/home/$user" of someone with lots of data and files

I know many efforts have been made to make the (meta)data operations "local" -- this speeds up spinning disks operations a lot, also SSD ones. However, having the whole metadata on an SSD disk (or a RAID1 of two such disks) could speed up many common tasks a lot. And the hardware price for such a benefit is really affordable now.

I see two possible implementations:

1. Re-work the Ext4 metadata operations (that work with inodes, etc) to read/write on a separate block device.

or

2. Add an option to the "data locality" algorithm to force it to store all metadata only at the beginning of a device (we can pre-allocate enough space). We can then transparently map in the DM those blocks to a separate faster block device, thus making the changes to Ext4 minimal.

Does all this make sense, or I'm missing something obvious?

Thank you for your time.

--
Best regards.
Ivan Zahariev | System Administrator | ICDSoft Ltd.

--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[Index of Archives]     [Reiser Filesystem Development]     [Ceph FS]     [Kernel Newbies]     [Security]     [Netfilter]     [Bugtraq]     [Linux FS]     [Yosemite National Park]     [MIPS Linux]     [ARM Linux]     [Linux Security]     [Linux RAID]     [Samba]     [Device Mapper]     [Linux Media]

  Powered by Linux