Hello, Here is some more info: At the moment is is a very small dataset. < 500 GB. I've got 4TB storage available in the server (2x 4TB in ZFS mirror) and I'm going to upgrade that to 12 TB netto in the near future, just to have more spindles available (4x 4TB in raidz1). When this 12TB is full (and I'm not sure at what rate this will go, could be full in 3 months or 2 years) I need to upgrade my setup to something else. And I'm looking at my options now, so I can have a plan ready on how to proceed at that point and I don't have to rush things. I'm using ZFS directly. It's DAS. The restore time isn't guaranteed. I want the backup solution to be HA, because when a backup is made from some system towards my storage, I need to have my service available. Because, like I said, this is the only backup available for a lot of the data stored on this server. Most of the source machines don't have any redundancy build in, so having backups is important. I don't have spare hardware on stock. I can arrange something when it goes horribly wrong, but it will impact my services in another way. It's not nice, but it will work. The risk of is not extremely high because of the age of the hardware. It isn't brand new, but it's not that old either. It's 2 years old now, and it has been running production for a year. But the PSU is single for instance. The machine has an Xeon E3-1220 V2 as CPU, so like I said: not that old :). I'm just storing backups on this server. I'm not backing up this server itself (yet ;). I just realized how stupid I am for not doing that, will be fixed in the very near future :p). Making a ZFS snapshot *and* have the DB consistent isn't that hard. You just run a FLUSH TABLES WITH READ LOCK command, ensuring that MySQL writes everything to the datafiles and locking the tables for writes. When you have the locks, this statement will be done. Then you have the system command in the mysql prompt, which you can use to run system command, so here you can let the mysql commandline tool start the snapshot process. And when that's done you just unlock the tables again. And in your snapshot the data is consistent. I don't have an example ready with this trick with a ZFS snapshot (because I don't backup the FreeBSD machine yet), but I do apply this trick on a CentOS 6 VM of mine. I'm making an LVM snapshot there instead of a ZFS snapshot, but the essence of the trick stays the same. This is how I do it there (replace the parts between <> with your own setting): mysql -h localhost -u root -p<mysql_root_password -e 'FLUSH TABLES WITH READ LOCK; SYSTEM /sbin/lvcreate --snapshot --size <snapshot_size> --name <snapshot_name> /dev/mapper/<vgname>-<lvname>; UNLOCK TABLES;' And for keeping the OwnCloud database in sync: Right after creating the snapshot and the clone I remove the entier cache for the specific user from the cache table in the OwnCloud database. I know this isn't recommended, but it works like a charm. But the cache in OwnCloud is something that keeps on bothering me, and even the removal of those lines isn't that great all the time. I need to redo the removal by hand, sometimes. And I don't like that. On Fri, June 26, 2015 11:58, 10 minus wrote: > Hi , > > As Christian has mentioned ... bit more detailed information will do us > good.. > Had explored Cephfs -- but performance was an issue vis-a-vis zfs when we > tested ( more than a year back) , so we did not get into details. > I will let the Cephfs experts chip in here on the present state of Cephfs > How are you using zfs on your main site .. nfs/cifs / iscsi > How much data are we talking about ? > Yes one machine is a SPOF but the questions you should ask or answer : > Is there a business requirement to restore data in a defined time ? > How much data is in play here ? > what are the odds it fails (hw quality is improving by the day -- does > mean > it wont fail) ? > How fast can one replace failed HW (We have spare HW always avaialbe) ? > do you need always on backup, especially offsite backup? > Have you explored Tape option ? > > We are using zfs on solaris and freebsd as a filer ( nfs/cifs) and we keep > three copies of snapshot (We have 5 TB of data ) > - local on filers ( snapshot every hour for 2 days) > - onsite on another machine 1 Week (snapshot copy every 12 hrs on a > machine onsite ) > - offsite (snapshot copy every day for 4 weeks --> then from offsite to > tape). > > For DB backup we have a system in place but it does not rely on zfs > snapshot, Would love to know how you manage DB backups with zfs snapshots. > ZFS is a mature technology .. > > > P.S We use ceph for openstack (ephemeral /cinder / glance ) .. with no > backup. (One year on we are still learning new things and it has just > worked) > > > > > On Fri, Jun 26, 2015 at 9:00 AM, Christian Balzer <chibi@xxxxxxx> wrote: > >> >> Hello, >> >> On Fri, 26 Jun 2015 00:28:20 +0200 Cybertinus wrote: >> >> > Hello everybody, >> > >> > >> > I'm looking at Ceph as an alternative for my current storage solution, >> > but I'm wondering if it is the right choice for me. I'm hoping you >> guys >> > can help me decide. >> > >> > The current setup is a FreeBSD 10.1 machine running entirely on ZFS. >> The >> > function of the machine is offsite backup for important data. For some >> > (fairly rapidly changing) data this server is the only backup of it. >> But >> > because the data is changing fairly quickly (every day at least) I'm >> > looking to get this server more HA then it is now. >> > It is just one FreeBSD machine, so this is an enormous SPOF off >> course. >> > >> But aside from the SPOF part that machine is sufficient for your usage, >> right? >> Care to share the specs of it and what data volume (total space used, >> daily >> transactions) we're talking about> >> >> > The most used functionality of ZFS that I use is the snapshot >> technology. >> > I've got multiple users on this server and each user has it's own >> > filesystem within the pool. And I just snapshot each filesystem >> regularly >> > and that way I enable the users to go back in time. >> > I've looked at the snapshot functionality of Ceph, but it's not clear >> to >> > me what I can snapshot with it exactly. >> > >> > Furthermore: what is the best way to hook Ceph to the application I >> use >> > to transfer the data from the users to the backup server? Today I'm >> using >> > OwnCloud, which is (in essence) a WebDAV server. Now I'm thinking >> about >> > replacing OwnCloud with something custom build. That way I can let PHP >> > talk directly with librados, which makes it easy to store the data. >> > Or I can keep on using OwnCloud and just hook up Ceph via CephFS. This >> > has the added advantage that I don't have to get my head around the >> > concept of object storage :p ;). >> > >> I'm slightly confused here, namely: >> You use owncloud (I got a test installation on a VM here, too), which >> uses a DB (mysql by default) to index the files uploaded. >> How do you make sure that your snapshots are consistent when it comes to >> DB files other than being lucky 99.9% of the time? >> >> I'll let the CephFS experts pipe up, but the usual disclaimers about >> CephFS stability do apply, in particular the latest (beta) version of >> Ceph >> has this line on top of the changelog: >> --- >> Highlights here include lots of RGW Swift fixes, RBD feature work >> surrounding the new object map feature, more CephFS snapshot fixes, and >> a >> few important CRUSH fixes. >> --- >> >> Now you could just mount an RBD image (or run a VM) with BTRFS and have >> snapshots again that are known to work. >> >> However going back to my first question up there, I have a feeling that >> a >> functional Ceph cluster with at least 3 storage nodes might be both too >> expensive while at the same time less performant than what you have now. >> >> A 2 node DRBD cluster might fit your needs better. >> >> Christian >> -- >> Christian Balzer Network/Systems Engineer >> chibi@xxxxxxx Global OnLine Japan/Fusion Communications >> http://www.gol.com/ >> _______________________________________________ >> ceph-users mailing list >> ceph-users@xxxxxxxxxxxxxx >> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com >> > _______________________________________________ > ceph-users mailing list > ceph-users@xxxxxxxxxxxxxx > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |