Hello everyone! Requesting for your comments and suggestions. We propose a new kernel module - blk-snap. This module implements snapshot and changed block tracking functionality. It is intended to create backup copies of any block devices without usage of device-mapper. Snapshots are temporary and are destroyed after the backup process has finished. Changed block tracking allows for incremental and differential backup copies. blk-snap uses block layer filter API. Block layer filter API provides a callback to intercept bio-requests. If a block device disappears for whatever reason, send a synchronous request to remove the device from filtering. Previously, we have already offered a patch with the implementation of the block layer filter api (https://patchwork.kernel.org/patch/11741447/). Link to that discussion: https://www.mail-archive.com/linux-kernel@xxxxxxxxxxxxxxx/msg2290127.html. Now we also offer the in-tree module. blk-snap kernel module is a product of a deep refactoring of the out-of-tree kernel veeamsnap (https://github.com/veeam/veeamsnap/) module: * all conditional compilation branches that served for the purpose of compatibility with older kernels have been removed; * linux kernel code style has been applied; * blk-snap mostly takes advantage of the existing kernel code instead of reinventing the wheel; * all redundant code (such as persistent cbt and snapstore collector) has been removed. We would appreciate your feedback! Several important things are still have to be done: * refactoring the module interface for interaction with a user-space code, it is already clear that the implementation of some calls can be improved; * haven't yet tested the build on architectures other than x86_64; * the user-space library is not ready yet and tool to control the module is to be created; * autotests need to be created; * regression testing has been conducted in a cursory manner, so there is a chance for mistakes. Sergei Shtepa (1): blk-snap - Block snapshot module This module implements snapshot and changed block tracking functionality. It is intended to create backup copies of any block devices without usage of device-mapper. MAINTAINERS | 14 + block/Kconfig | 11 + block/Makefile | 1 + block/blk-core.c | 11 +- block/blk-filter-internal.h | 34 + block/blk-filter.c | 288 ++++++ block/genhd.c | 24 + drivers/block/Kconfig | 6 +- drivers/block/Makefile | 3 +- drivers/block/blk-snap/Kconfig | 24 + drivers/block/blk-snap/Makefile | 29 + drivers/block/blk-snap/big_buffer.c | 193 ++++ drivers/block/blk-snap/big_buffer.h | 24 + drivers/block/blk-snap/blk-snap-ctl.h | 190 ++++ drivers/block/blk-snap/blk_deferred.c | 566 +++++++++++ drivers/block/blk-snap/blk_deferred.h | 67 ++ drivers/block/blk-snap/blk_descr_file.c | 82 ++ drivers/block/blk-snap/blk_descr_file.h | 26 + drivers/block/blk-snap/blk_descr_mem.c | 66 ++ drivers/block/blk-snap/blk_descr_mem.h | 14 + drivers/block/blk-snap/blk_descr_multidev.c | 86 ++ drivers/block/blk-snap/blk_descr_multidev.h | 25 + drivers/block/blk-snap/blk_descr_pool.c | 190 ++++ drivers/block/blk-snap/blk_descr_pool.h | 38 + drivers/block/blk-snap/blk_redirect.c | 507 ++++++++++ drivers/block/blk-snap/blk_redirect.h | 73 ++ drivers/block/blk-snap/blk_util.c | 33 + drivers/block/blk-snap/blk_util.h | 10 + drivers/block/blk-snap/cbt_map.c | 210 +++++ drivers/block/blk-snap/cbt_map.h | 62 ++ drivers/block/blk-snap/common.h | 29 + drivers/block/blk-snap/ctrl_fops.c | 693 ++++++++++++++ drivers/block/blk-snap/ctrl_fops.h | 19 + drivers/block/blk-snap/ctrl_pipe.c | 562 +++++++++++ drivers/block/blk-snap/ctrl_pipe.h | 34 + drivers/block/blk-snap/ctrl_sysfs.c | 73 ++ drivers/block/blk-snap/ctrl_sysfs.h | 5 + drivers/block/blk-snap/defer_io.c | 393 ++++++++ drivers/block/blk-snap/defer_io.h | 39 + drivers/block/blk-snap/filter.c | 72 ++ drivers/block/blk-snap/filter.h | 7 + drivers/block/blk-snap/main.c | 83 ++ drivers/block/blk-snap/params.c | 57 ++ drivers/block/blk-snap/params.h | 29 + drivers/block/blk-snap/rangevector.c | 85 ++ drivers/block/blk-snap/rangevector.h | 31 + drivers/block/blk-snap/snapimage.c | 982 ++++++++++++++++++++ drivers/block/blk-snap/snapimage.h | 16 + drivers/block/blk-snap/snapshot.c | 228 +++++ drivers/block/blk-snap/snapshot.h | 17 + drivers/block/blk-snap/snapstore.c | 929 ++++++++++++++++++ drivers/block/blk-snap/snapstore.h | 68 ++ drivers/block/blk-snap/snapstore_device.c | 532 +++++++++++ drivers/block/blk-snap/snapstore_device.h | 63 ++ drivers/block/blk-snap/snapstore_file.c | 52 ++ drivers/block/blk-snap/snapstore_file.h | 15 + drivers/block/blk-snap/snapstore_mem.c | 89 ++ drivers/block/blk-snap/snapstore_mem.h | 20 + drivers/block/blk-snap/snapstore_multidev.c | 118 +++ drivers/block/blk-snap/snapstore_multidev.h | 22 + drivers/block/blk-snap/tracker.c | 521 +++++++++++ drivers/block/blk-snap/tracker.h | 50 + drivers/block/blk-snap/tracking.c | 260 ++++++ drivers/block/blk-snap/tracking.h | 12 + drivers/block/blk-snap/version.h | 7 + include/linux/blk-filter.h | 41 + include/linux/genhd.h | 3 +- 67 files changed, 9157 insertions(+), 6 deletions(-) create mode 100644 block/blk-filter-internal.h create mode 100644 block/blk-filter.c create mode 100644 drivers/block/blk-snap/Kconfig create mode 100644 drivers/block/blk-snap/Makefile create mode 100644 drivers/block/blk-snap/big_buffer.c create mode 100644 drivers/block/blk-snap/big_buffer.h create mode 100644 drivers/block/blk-snap/blk-snap-ctl.h create mode 100644 drivers/block/blk-snap/blk_deferred.c create mode 100644 drivers/block/blk-snap/blk_deferred.h create mode 100644 drivers/block/blk-snap/blk_descr_file.c create mode 100644 drivers/block/blk-snap/blk_descr_file.h create mode 100644 drivers/block/blk-snap/blk_descr_mem.c create mode 100644 drivers/block/blk-snap/blk_descr_mem.h create mode 100644 drivers/block/blk-snap/blk_descr_multidev.c create mode 100644 drivers/block/blk-snap/blk_descr_multidev.h create mode 100644 drivers/block/blk-snap/blk_descr_pool.c create mode 100644 drivers/block/blk-snap/blk_descr_pool.h create mode 100644 drivers/block/blk-snap/blk_redirect.c create mode 100644 drivers/block/blk-snap/blk_redirect.h create mode 100644 drivers/block/blk-snap/blk_util.c create mode 100644 drivers/block/blk-snap/blk_util.h create mode 100644 drivers/block/blk-snap/cbt_map.c create mode 100644 drivers/block/blk-snap/cbt_map.h create mode 100644 drivers/block/blk-snap/common.h create mode 100644 drivers/block/blk-snap/ctrl_fops.c create mode 100644 drivers/block/blk-snap/ctrl_fops.h create mode 100644 drivers/block/blk-snap/ctrl_pipe.c create mode 100644 drivers/block/blk-snap/ctrl_pipe.h create mode 100644 drivers/block/blk-snap/ctrl_sysfs.c create mode 100644 drivers/block/blk-snap/ctrl_sysfs.h create mode 100644 drivers/block/blk-snap/defer_io.c create mode 100644 drivers/block/blk-snap/defer_io.h create mode 100644 drivers/block/blk-snap/filter.c create mode 100644 drivers/block/blk-snap/filter.h create mode 100644 drivers/block/blk-snap/main.c create mode 100644 drivers/block/blk-snap/params.c create mode 100644 drivers/block/blk-snap/params.h create mode 100644 drivers/block/blk-snap/rangevector.c create mode 100644 drivers/block/blk-snap/rangevector.h create mode 100644 drivers/block/blk-snap/snapimage.c create mode 100644 drivers/block/blk-snap/snapimage.h create mode 100644 drivers/block/blk-snap/snapshot.c create mode 100644 drivers/block/blk-snap/snapshot.h create mode 100644 drivers/block/blk-snap/snapstore.c create mode 100644 drivers/block/blk-snap/snapstore.h create mode 100644 drivers/block/blk-snap/snapstore_device.c create mode 100644 drivers/block/blk-snap/snapstore_device.h create mode 100644 drivers/block/blk-snap/snapstore_file.c create mode 100644 drivers/block/blk-snap/snapstore_file.h create mode 100644 drivers/block/blk-snap/snapstore_mem.c create mode 100644 drivers/block/blk-snap/snapstore_mem.h create mode 100644 drivers/block/blk-snap/snapstore_multidev.c create mode 100644 drivers/block/blk-snap/snapstore_multidev.h create mode 100644 drivers/block/blk-snap/tracker.c create mode 100644 drivers/block/blk-snap/tracker.h create mode 100644 drivers/block/blk-snap/tracking.c create mode 100644 drivers/block/blk-snap/tracking.h create mode 100644 drivers/block/blk-snap/version.h create mode 100644 include/linux/blk-filter.h -- 2.28.0
