Basic vfs-level fiemap infrastructure, which sets up a new ->fiemap
inode operation.
Userspace can get extent information on a file via fiemap ioctl. As input,
the fiemap ioctl takes a struct fiemap which includes an array of struct
fiemap_extent (fm_extents). Size of the extent array is passed as
fm_extent_count and number of extents returned will be written into
fm_mapped_extents. Offset and length fields on the fiemap structure
(fm_start, fm_length) describe a logical range which will be searched for
extents. All extents returned will at least partially contain this range.
The actual extent offsets and ranges returned will be unmodified from their
offset and range on-disk.
The fiemap ioctl returns '0' on success. On error, -1 is returned and errno
is set. If errno is equal to EBADR, then fm_flags will contain those flags
which were passed in which the kernel did not understand. On all other
errors, the contents of fm_extents is undefined.
As fiemap evolved, there have been many authors of the vfs patch. As far as
I can tell, the list includes:
Kalpak Shah <kalpak.shah@xxxxxxx>
Andreas Dilger <adilger@xxxxxxx>
Eric Sandeen <sandeen@xxxxxxxxxx>
Mark Fasheh <mfasheh@xxxxxxxx>
Signed-off-by: Mark Fasheh <mfasheh@xxxxxxxx>
---
Documentation/filesystems/fiemap.txt | 224 ++++++++++++++++++++++++++++++++++
fs/ioctl.c | 140 +++++++++++++++++++++
include/linux/fiemap.h | 72 +++++++++++
include/linux/fs.h | 16 +++
4 files changed, 452 insertions(+), 0 deletions(-)
create mode 100644 Documentation/filesystems/fiemap.txt
create mode 100644 include/linux/fiemap.h
diff --git a/Documentation/filesystems/fiemap.txt b/Documentation/filesystems/fiemap.txt
new file mode 100644
index 0000000..84acf6c
--- /dev/null
+++ b/Documentation/filesystems/fiemap.txt
@@ -0,0 +1,224 @@
+Fiemap Ioctl
+============
+
+The fiemap ioctl is an efficient method for userspace to get file
+extent mappings. Instead of block-by-block mapping (such as bmap), fiemap
+returns a list of extents.
+
+
+Request Basics
+--------------
+
+A fiemap request is encoded within struct fiemap:
+
+struct fiemap {
+ __u64 fm_start; /* logical offset (inclusive) at
+ * which to start mapping (in) */
+ __u64 fm_length; /* logical length of mapping which
+ * userspace cares about (in) */
+ __u32 fm_flags; /* FIEMAP_FLAG_* flags for request (in) */
+ __u32 fm_extent_count; /* size of fm_extents array (in) */
+ __u32 fm_mapped_extents; /* number of extents that were
+ * mapped (out) */
+ __u32 fm_reserved;
+ struct fiemap_extent fm_extents[0];
+};
+
+
+fm_start, and fm_length specify the logical range within the file
+which the process would like mappings for. Extents returned mirror
+those on disk - that is, the logical offset of the 1st returned extent
+may start before fm_start, and the range covered by the last returned
+extent may end after fm_length. All offsets and lengths are in bytes.
+
+Certain flags to modify the way in which mappings are looked up can be
+set in fm_flags. If the kernel doesn't understand some particular
+flags, it will return EBADR and the contents of fm_flags will contain
+the set of flags which caused the error. If the kernel is compatible
+with all flags passed, the contents of fm_flags will be unmodified.
+It is up to userspace to determine whether rejection of a particular
+flag is fatal to it's operation. This scheme is intended to allow the
+fiemap interface to grow in the future but without losing
+compatibility with old software.
+
+Currently, there are four flags which can be set in fm_flags:
+
+* FIEMAP_FLAG_NUM_EXTENTS
+If this flag is set, extent information will not be returned via the
+fm_extents array and the value of fm_extent_count will be
+ignored. Instead, the total number of extents covering the range will
+be returned via fm_mapped_extents. This is useful for programs which
+only want to count the number of extents in a file, but don't care
+about the actual extent layout.
+
+* FIEMAP_FLAG_SYNC
+If this flag is set, the kernel will sync the file before mapping extents.
+
+* FIEMAP_FLAG_HSM_READ
+If the extent is offline, retrieve it before mapping and do not flag
+it as FIEMAP_EXTENT_SECONDARY. This flag has no effect if the file
+system does not support HSM.
+
+* FIEMAP_FLAG_XATTR
+If this flag is set, the extents returned will describe the inodes
+extended attribute lookup tree, instead of it's data tree.
+
+* FIEMAP_FLAG_LUN_ORDER
+If the file system stripes file data, this will return contiguous
+regions of physical allocation, sorted by LUN. Logical offsets may not
+make sense if this flag is passed. If the file system does not support
+multiple LUNs, this flag will be ignored.
+
+
+Extent Mapping
+--------------
+
+Note that all of this is ignored if FIEMAP_FLAG_NUM_EXTENTS is set.
+
+Extent information is returned within the embedded fm_extents array
+which userspace must allocate along with the fiemap structure. The
+total number of fiemap_extents available should be passed via
+fm_extent_count. The of extents mapped by kernel will be returned via
+fm_mapped_extents. If the number of fiemap_extents allocated is less
+than would be required to map the requested range, the maximum number
+of extents that can be mapped in available memory will be returned and
+fm_mapped_extents will be equal to fm_extent_count. In that case, the
+last extent in the array will not complete the requested range and
+will not have the FIEMAP_EXTENT_LAST flag set (see the next section on
+extent flags).
+
+Each extent is described by a single fiemap_extent structure as
+returned in fm_extents.
+
+struct fiemap_extent {
+ __u64 fe_logical;/* logical offset in bytes for the start of
+ * the extent */
+ __u64 fe_physical; /* physical offset in bytes for the start
+ * of the extent */
+ __u64 fe_length; /* length in bytes for the extent */
+ __u32 fe_flags; /* returned FIEMAP_EXTENT_* flags for the extent */
+ __u32 fe_lun; /* logical device number for extent (starting at 0)*/
+};
+
+All offsets and lengths are in bytes and mirror those on disk - it is
+valid for an extents logical offset to start before the request or
+it's logical length to extend past the request. Unless
+FIEMAP_EXTENT_NOT_ALIGNED is returned, fe_logical, fe_physical and
+fe_length will be aligned to the block size of the file system.
+
+The fe_flags field contains flags which describe the extent
+returned. A special flag, FIEMAP_EXTENT_LAST is always set on the last
+extent in the file so that the process making fiemap calls can
+determine when no more extents are available.
+
+Some flags are intentionally vague and will always be set in the
+presence of other more specific flags. This way a program looking for
+a general property does not have to know all existing and future flags
+which imply that property.
+
+For example, if FIEMAP_EXTENT_DATA_INLINE or FIEMAP_EXTENT_DATA_TAIL
+are set, FIEMAP_EXTENT_NOT_ALIGNED will also be set. A program looking
+for inline or tail-packed data can key on the specific flag. Software
+which simply cares not to try operating on non-aligned extents
+however, can just key on FIEMAP_EXTENT_NOT_ALIGNED, and not have to
+worry about all present and future flags which might imply unaligned
+data. Note that the opposite is not true - it would be valid for
+FIEMAP_EXTENT_NOT_ALIGNED to appear alone.
+
+* FIEMAP_EXTENT_LAST
+This is the last extent in the file. A mapping attempt past this
+extent will return nothing.
+
+* FIEMAP_EXTENT_UNKNOWN
+The location of this extent is currently unknown. This may indicate
+the data is stored on an inaccessible volume or that no storage has
+been allocated for the file yet.
+
+* FIEMAP_EXTENT_SECONDARY
+ - This will also set FIEMAP_EXTENT_UNKNOWN.
+The data for this extent is in secondary storage.
+
+* FIEMAP_EXTENT_DELALLOC
+ - This will also set FIEMAP_EXTENT_UNKNOWN.
+Delayed allocation - while there is data for this extent, it's
+physical location has not been allocated yet.
+
+* FIEMAP_EXTENT_NO_DIRECT
+Direct access to the data in this extent is illegal or will have
+undefined results.
+
+* FIEMAP_EXTENT_NET
+ - This will also set FIEMAP_EXTENT_NO_DIRECT
+The data for this extent is not stored in a locally-accessible device.
+
+* FIEMAP_EXTENT_DATA_COMPRESSED
+ - This will also set FIEMAP_EXTENT_NO_DIRECT
+The data in this extent has been compressed by the file system.
+
+* FIEMAP_EXTENT_DATA_ENCRYPTED
+ - This will also set FIEMAP_EXTENT_NO_DIRECT
+The data in this extent has been encrypted by the file system.
+
+* FIEMAP_EXTENT_NOT_ALIGNED
+Extent offsets and length are not guaranteed to be block aligned.
+
+* FIEMAP_EXTENT_DATA_INLINE
+ This will also set FIEMAP_EXTENT_NOT_ALIGNED
+Data is located within a meta data block.
+
+* FIEMAP_EXTENT_DATA_TAIL
+ This will also set FIEMAP_EXTENT_NOT_ALIGNED
+Data is packed into a block with data from other files.
+
+* FIEMAP_EXTENT_UNWRITTEN
+Unwritten extent - the extent is allocated but it's data has not been
+initialized.
+
+
+VFS -> File System Implementation
+---------------------------------
+
+File systems wishing to support fiemap must implement a ->fiemap
+callback (on struct inode_operations):
+
+struct inode_operations {
+ ...
+
+ int (*fiemap) (struct inode *, struct fiemap_extent_info *, u64 start,
+ u64 len);
+
+->fiemap is passed struct fiemap_extent_info which describes the
+fiemap request:
+
+struct fiemap_extent_info {
+ unsigned int fi_flags; /* Flags as passed from user */
+ unsigned int fi_extents_mapped; /* Number of mapped extents */
+ unsigned int fi_extents_max; /* Size of fiemap_extent array */
+ char *fi_extents_start; /* Start of fiemap_extent array */
+};
+
+It is intended that the file system should only need to access
+fi_flags directly. Aside from checking fi_flags to modify callback
+behavior, flags which the file system can not handle, can be written
+into fieinfo->fi_flags. In this case, the file system *must* return
+-EBADR so that ioctl_fiemap() can write them into the userspace
+buffer.
+
+For each extent in the request range, the file system should call
+the helper function, fiemap_fill_next_extent():
+
+int fiemap_fill_next_extent(struct fiemap_extent_info *info, u64 logical,
+ u64 phys, u64 len, u32 flags, u32 lun);
+
+fiemap_fill_next_extent() will use the passed values to populate the
+next free extent in the fm_extents array. 'General' extent flags will
+automatically be set from specific flags on behalf of the calling file
+system so that the userspace API is not broken.
+
+fiemap_fill_next_extent() returns 0 on success, and 1 when the
+user-supplied fm_extents array is full. If an error is encountered
+while copying the extent to user memory, -EFAULT will be returned.
+
+If the request has the FIEMAP_FLAG_NUM_EXTENTS flag set, then calling
+this helper is not necessary and fi_extents_mapped can be set
+directly.
diff --git a/fs/ioctl.c b/fs/ioctl.c
index 7db32b3..405bbcb 100644
--- a/fs/ioctl.c
+++ b/fs/ioctl.c
@@ -16,6 +16,9 @@
#include <asm/ioctls.h>
+/* So that the fiemap access checks can't overflow on 32 bit machines. */
+#define FIEMAP_MAX_EXTENTS (UINT_MAX / sizeof(struct fiemap_extent))
+
/**
* vfs_ioctl - call filesystem specific ioctl methods
* @filp: open file to invoke ioctl method on
@@ -71,6 +74,141 @@ static int ioctl_fibmap(struct file *filp, int __user *p)
return put_user(res, p);
}
+/**
+ * fiemap_fill_next_extent - Fiemap helper function
+ * @fieinfo: Fiemap context passed into ->fiemap
+ * @logical: Extent logical start offset, in bytes
+ * @phys: Extent physical start offset, in bytes
+ * @len: Extent length, in bytes
+ * @flags: FIEMAP_EXTENT flags that describe this extent
+ * @lun: LUN on which this extent resides
+ *
+ * Called from file system ->fiemap callback. Will populate extent
+ * info as passed in via arguments and copy to user memory. On
+ * success, extent count on fieinfo is incremented.
+ *
+ * Returns 0 on success, -errno on error, 1 if this was the last
+ * extent that will fit in user array.
+ */
+#define SET_UNKNOWN_FLAGS (FIEMAP_EXTENT_SECONDARY|FIEMAP_EXTENT_DELALLOC)
+#define SET_NO_DIRECT_FLAGS (FIEMAP_EXTENT_DATA_COMPRESSED \
+ |FIEMAP_EXTENT_DATA_ENCRYPTED \
+ |FIEMAP_EXTENT_NET)
+#define SET_NOT_ALIGNED_FLAGS (FIEMAP_EXTENT_DATA_TAIL|FIEMAP_EXTENT_DATA_INLINE)
+int fiemap_fill_next_extent(struct fiemap_extent_info *fieinfo,
+ u64 logical, u64 phys, u64 len, u32 flags, u32 lun)
+{
+ struct fiemap_extent extent;
+ char *dest = fieinfo->fi_extents_start;
+
+ if (fieinfo->fi_flags & FIEMAP_FLAG_NUM_EXTENTS) {
+ fieinfo->fi_extents_mapped++;
+ return (flags & FIEMAP_EXTENT_LAST) ? 1 : 0;
+ }
+
+ if (fieinfo->fi_extents_mapped >= fieinfo->fi_extents_max)
+ return 1;
+
+ if (flags & SET_UNKNOWN_FLAGS)
+ flags |= FIEMAP_EXTENT_UNKNOWN;
+ if (flags & SET_NO_DIRECT_FLAGS)
+ flags |= FIEMAP_EXTENT_NO_DIRECT;
+ if (flags & SET_NOT_ALIGNED_FLAGS)
+ flags |= FIEMAP_EXTENT_NOT_ALIGNED;
+
+ extent.fe_logical = logical;
+ extent.fe_physical = phys;
+ extent.fe_length = len;
+ extent.fe_flags = flags;
+ extent.fe_lun = lun;
+
+ dest += (fieinfo->fi_extents_mapped * sizeof(extent));
+ if (copy_to_user(dest, &extent, sizeof(extent)))
+ return -EFAULT;
+
+ fieinfo->fi_extents_mapped++;
+ if (fieinfo->fi_extents_mapped == fieinfo->fi_extents_max)
+ return 1;
+ return 0;
+}
+EXPORT_SYMBOL(fiemap_fill_next_extent);
+
+static int ioctl_fiemap(struct file *filp, unsigned long arg)
+{
+ struct fiemap fiemap;
+ u64 len;
+ u32 incompat_flags;
+ struct fiemap_extent_info fieinfo = {0, };
+ struct inode *inode = filp->f_path.dentry->d_inode;
+ struct super_block *sb = inode->i_sb;
+ int error = 0;
+
+ if (!inode->i_op->fiemap)
+ return -EOPNOTSUPP;
+
+ if (copy_from_user(&fiemap, (struct fiemap __user *) arg,
+ sizeof(struct fiemap)))
+ return -EFAULT;
+
+ /*
+ * The VFS does basic sanity checks on the flag
+ * value. Individual file systems can also reject otherwise
+ * 'valid' flags by returning -EBADR from ->fiemap.
+ */
+ incompat_flags = fiemap.fm_flags & ~FIEMAP_FLAGS_COMPAT;
+ if (incompat_flags)
+ goto out_bad_flags;
+
+ if (!(fiemap.fm_flags & FIEMAP_FLAG_NUM_EXTENTS) &&
+ (fiemap.fm_extent_count == 0 ||
+ fiemap.fm_extent_count > FIEMAP_MAX_EXTENTS))
+ return -EINVAL;
+
+ if (fiemap.fm_start > sb->s_maxbytes)
+ return -EFBIG;
+
+ len = fiemap.fm_length;
+ if (len == 0)
+ return -EINVAL;
+ /*
+ * Shrink request scope to what the fs can actually handle.
+ */
+ if ((len > sb->s_maxbytes) ||
+ (sb->s_maxbytes - len) < fiemap.fm_start)
+ len = sb->s_maxbytes - fiemap.fm_start;
+
+ fieinfo.fi_flags = fiemap.fm_flags;
+ if (!(fiemap.fm_flags & FIEMAP_FLAG_NUM_EXTENTS)) {
+ fieinfo.fi_extents_max = fiemap.fm_extent_count;
+ fieinfo.fi_extents_start = (char *)arg + sizeof(fiemap);
+
+ if (!access_ok(VERIFY_WRITE, fieinfo.fi_extents_start,
+ fieinfo.fi_extents_max * sizeof(struct fiemap_extent)))
+ return -EFAULT;
+ }
+
+ if (fieinfo.fi_flags & FIEMAP_FLAG_SYNC)
+ filemap_write_and_wait(inode->i_mapping);
+
+ error = inode->i_op->fiemap(inode, &fieinfo, fiemap.fm_start, len);
+ if (error == -EBADR) {
+ incompat_flags = fieinfo.fi_flags;
+ goto out_bad_flags;
+ }
+ if (error == 0) {
+ fiemap.fm_mapped_extents = fieinfo.fi_extents_mapped;
+ error = copy_to_user((char *)arg, &fiemap, sizeof(fiemap));
+ }
+
+ return error;
+
+out_bad_flags:
+ fiemap.fm_flags = incompat_flags;
+ if (copy_to_user((char *)arg, &fiemap, sizeof(fiemap)))
+ return -EFAULT;
+ return -EBADR;
+}
+
static int file_ioctl(struct file *filp, unsigned int cmd,
unsigned long arg)
{
@@ -80,6 +218,8 @@ static int file_ioctl(struct file *filp, unsigned int cmd,
switch (cmd) {
case FIBMAP:
return ioctl_fibmap(filp, p);
+ case FS_IOC_FIEMAP:
+ return ioctl_fiemap(filp, arg);
case FIGETBSZ:
return put_user(inode->i_sb->s_blocksize, p);
case FIONREAD:
diff --git a/include/linux/fiemap.h b/include/linux/fiemap.h
new file mode 100644
index 0000000..a6a629a
--- /dev/null
+++ b/include/linux/fiemap.h
@@ -0,0 +1,72 @@
+/*
+ * FIEMAP ioctl infrastructure.
+ *
+ * Copyright (C) 2007 Cluster File Systems, Inc
+ *
+ * Author: Kalpak Shah <kalpak.shah@xxxxxxx>
+ * Andreas Dilger <adilger@xxxxxxx>
+ */
+
+#ifndef _LINUX_FIEMAP_H
+#define _LINUX_FIEMAP_H
+
+struct fiemap_extent {
+ __u64 fe_logical;/* logical offset in bytes for the start of
+ * the extent from the beginning of the file */
+ __u64 fe_physical; /* physical offset in bytes for the start
+ * of the extent from the beginning of the disk */
+ __u64 fe_length; /* length in bytes for the extent */
+ __u32 fe_flags; /* returned FIEMAP_EXTENT_* flags for the extent */
+ __u32 fe_lun; /* logical device number for extent (starting at 0)*/
+};
+
+struct fiemap {
+ __u64 fm_start; /* logical offset (inclusive) at
+ * which to start mapping (in) */
+ __u64 fm_length; /* logical length of mapping which
+ * userspace cares about (in) */
+ __u32 fm_flags; /* FIEMAP_FLAG_* flags for request (in/out) */
+ __u32 fm_extent_count; /* size of fm_extents array (in) */
+ __u32 fm_mapped_extents; /* number of extents that were
+ * mapped (out) */
+ __u32 fm_reserved;
+ struct fiemap_extent fm_extents[0];
+};
+
+#define FIEMAP_MAX_OFFSET (~0ULL)
+
+#define FIEMAP_FLAG_NUM_EXTENTS 0x00000001 /* return only number of extents */
+#define FIEMAP_FLAG_SYNC 0x00000002 /* sync file data before map */
+#define FIEMAP_FLAG_HSM_READ 0x00000004 /* get data from HSM before map */
+#define FIEMAP_FLAG_XATTR 0x00000008 /* map extended attribute tree */
+#define FIEMAP_FLAG_LUN_ORDER 0x00000010 /* return lun-sorted mappings */
+
+#define FIEMAP_FLAGS_COMPAT (FIEMAP_FLAG_NUM_EXTENTS|FIEMAP_FLAG_SYNC| \
+ FIEMAP_FLAG_HSM_READ|FIEMAP_FLAG_XATTR| \
+ FIEMAP_FLAG_LUN_ORDER)
+
+#define FIEMAP_EXTENT_LAST 0x00000001 /* Last extent in file. */
+#define FIEMAP_EXTENT_UNKNOWN 0x00000002 /* In use, location unknown. */
+#define FIEMAP_EXTENT_SECONDARY 0x00000004 /* Data in secondary storage.
+ * Sets EXTENT_UNKNOWN. */
+#define FIEMAP_EXTENT_DELALLOC 0x00000008 /* Has data, but not yet
+ * written.
+ * Sets EXTENT_UNKNOWN. */
+#define FIEMAP_EXTENT_NO_DIRECT 0x00000010 /* Cannot access data directly. */
+#define FIEMAP_EXTENT_NET 0x00000020 /* Data stored remotely.
+ * Sets EXTENT_NO_DIRECT. */
+#define FIEMAP_EXTENT_DATA_COMPRESSED 0x00000040 /* Data is compressed by fs.
+ * Sets EXTENT_NO_DIRECT. */
+#define FIEMAP_EXTENT_DATA_ENCRYPTED 0x00000080 /* Data is encrypted by fs.
+ * Sets EXTENT_NO_DIRECT. */
+#define FIEMAP_EXTENT_NOT_ALIGNED 0x00000100 /* Extent offsets may not be
+ * block aligned. */
+#define FIEMAP_EXTENT_DATA_INLINE 0x00000200 /* Data stored within fs
+ * metadata.
+ * Sets EXTENT_NOT_ALIGNED.*/
+#define FIEMAP_EXTENT_DATA_TAIL 0x00000400 /* Data is tail-packed.
+ * Sets EXTENT_NOT_ALIGNED.*/
+#define FIEMAP_EXTENT_UNWRITTEN 0x00000800 /* Space allocated, but
+ * no data. */
+
+#endif /* _LINUX_FIEMAP_H */
diff --git a/include/linux/fs.h b/include/linux/fs.h
index f413085..ed90393 100644
--- a/include/linux/fs.h
+++ b/include/linux/fs.h
@@ -228,6 +228,7 @@ extern int dir_notify_enable;
#define FS_IOC_SETFLAGS _IOW('f', 2, long)
#define FS_IOC_GETVERSION _IOR('v', 1, long)
#define FS_IOC_SETVERSION _IOW('v', 2, long)
+#define FS_IOC_FIEMAP _IOWR('f', 11, struct fiemap)
#define FS_IOC32_GETFLAGS _IOR('f', 1, int)
#define FS_IOC32_SETFLAGS _IOW('f', 2, int)
#define FS_IOC32_GETVERSION _IOR('v', 1, int)
@@ -288,6 +289,7 @@ extern int dir_notify_enable;
#include <linux/mutex.h>
#include <linux/capability.h>
#include <linux/semaphore.h>
+#include <linux/fiemap.h>
#include <asm/atomic.h>
#include <asm/byteorder.h>
@@ -1144,6 +1146,18 @@ extern void dentry_unhash(struct dentry *dentry);
extern int file_permission(struct file *, int);
/*
+ * VFS FIEMAP helper definitions.
+ */
+struct fiemap_extent_info {
+ unsigned int fi_flags; /* Flags as passed from user */
+ unsigned int fi_extents_mapped; /* Number of mapped extents */
+ unsigned int fi_extents_max; /* Size of fiemap_extent array */
+ char *fi_extents_start; /* Start of fiemap_extent array */
+};
+int fiemap_fill_next_extent(struct fiemap_extent_info *info, u64 logical,
+ u64 phys, u64 len, u32 flags, u32 lun);
+
+/*
* File types
*
* NOTE! These match bits 12..15 of stat.st_mode
@@ -1273,6 +1287,8 @@ struct inode_operations {
void (*truncate_range)(struct inode *, loff_t, loff_t);
long (*fallocate)(struct inode *inode, int mode, loff_t offset,
loff_t len);
+ int (*fiemap) (struct inode *, struct fiemap_extent_info *, u64 start,
+ u64 len);
};
struct seq_file;
--
1.5.4.1
--
To unsubscribe from this list: send the line "unsubscribe linux-fsdevel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at http://vger.kernel.org/majordomo-info.html
