Please see: http://tracker.ceph.com/issues/4084 rbd: incremental backups http://tracker.ceph.com/issues/3387 librbd: expose changed objects since a given snapshot http://tracker.ceph.com/issues/3272 send/receive rbd snapshots It would be great if we could track this discussion in those tickets. Ian R. Colle Ceph Program Manager Inktank Cell: +1.303.601.7713 <tel:%2B1.303.601.7713> Email: ian@xxxxxxxxxxx <http://www.linkedin.com/in/ircolle> <http://www.twitter.com/ircolle> On 2/28/13 4:33 PM, "Josh Durgin" <josh.durgin@xxxxxxxxxxx> wrote: >On 02/16/2013 03:51 AM, Jens Kristian Søgaard wrote: >> Hi Sage, >> >>> 1) Decide what output format to use. We want to use something that is >> >> I have given it some thought, and my initial suggestion to keep things >> simple is to use the QCOW2 image format. >> >> The birds eye view of the process would be as follows: >> >> * Initial backup >> >> User supplied information: pool, image name >> >> Create rbd snapshot of the image named "backup_1", where 1 could be a >> timestamp or an integer count. >> >> Save the snapshot to a standard qcow2 image. Similar to: >> >> qemu-img convert rbd:data/myimage@backup_1 -O qcow2 >> data_myimage_backup_1.qcow2 >> >> Note: I don't know if qemu-img actually supports reading from snapshots >> currently. > >It does. > >> * Incremental backup >> >> User supplied information: pool, image name, path to initial backup or >> previous incremental file >> >> Create rbd snapshot of the image named "backup_2", where 2 could be a >> timestamp or an integer count. >> >> Determine previous snapshot identifier from given file name. >> >> Determine objects changed from the snapshot given by that identifier and >> the newly created snapshot. >> >> Construct QCOW2 L1- and L2-tables in memory from that changeset. >> >> Create new qcow2 image with the previous backup file as the backing >> image, and write out the tables and changed blocks. >> >> Delete previous rbd snapshot. >> >> >> * Restoring and mounting >> >> >> The use of the QCOW2 format means that we can use existing tools for >> restoring and mounting the backups. >> >> To restore a backup the user can simply choose either the initial backup >> file or an incremental, and use qemu-img to copy that to a new rbd >>image. >> >> To mount the initial backup or an incremental, the user can use qemu-nbd >> to mount and explore the backup to determine which one to restore. >> >> The performance of restores and mounts would ofcourse be weakened if the >> backup consists of a large number of incrementals. In that case the >> existing qemu-img tool could be used to flatten the backup. >> >> >> * Pros/cons >> >> The QCOW2 format support compression, so we could implement compressed >> backups without much effort. >> >> The disadvantages to using QCOW2 like this is that we do not have any >> checksumming or safe guards against potential errors such as users >> mixing up images. >> >> Another disadvantage to this approach is that vital information is >> stored in the actual filename of the backup file. I don't see any place >> in the QCOW2 file format for storing this information inside the file, >> sadly. >> >> We could opt for storing it inside a plain text file that accompanies >> the QCOW2 file, or tarballing the qcow2 file and that plain text file. > >qcow2 seems like a good initial format given the existing tools. We >could always add another format later, or wrap it with extra >information like you suggest. > >Have you had a chance to start implementing this yet? It'd be great to >get it working in the next month. > >Josh >-- >To unsubscribe from this list: send the line "unsubscribe ceph-devel" in >the body of a message to majordomo@xxxxxxxxxxxxxxx >More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
