Gluster may be slow when creating lots of small files but it is not slow writing. I don't see a problem with what you want to do as long as you realise that many of the files will be out of place and a future rebalance would take a very long time - if you decide to run one. On Wed, 2014-10-15 at 21:12 -0500, Ryan Nix wrote: > Interesting. Still, I think its better to let the Gluster client > handle the syncing. What happens if, for some strange reason, the > rsync process dies in the middle of the night? Gluster, on the other, > will keep working to get the data on the other bricks without human > intervention. I recently used Gluster to sync 3 TBs of data to the > another brick over a 1Gbps link in about 13 hours on decent hardware. > > On Wed, Oct 15, 2014 at 9:04 PM, SINCOCK John <J.Sincock@xxxxxxxxx> > wrote: > > > We have 20 Terabytes to rsync onto a new server (which will > have 32 TB capacity), > > And we then want to add that server to an existing 2-node > gluster of 73TB (53 TB used, 20 TB free), to give a 3-node > gluster with 105TB capacity, 73TB used. > > > > The reason I want to do it this way, if possible, is that > Gluster is slow on writes, especially for small files, and we > have a LOT of small files, so I’m pretty sure it will be LOT > faster to rsync directly to the new server (which is the one > that has free space anyway), and then add that server to the > gluster – if it is possible to have gluster recognise those > files. > > > > > > From: Ryan Nix [mailto:ryan.nix@xxxxxxxxx] > Sent: Thursday, 16 October 2014 11:58 AM > To: SINCOCK John > Cc: Franco Broi; gluster-users > > > Subject: Re: Is it ok to add a new brick with > files already on it? > > > So Gluster, at its core, uses rsync to copy the data to the > other bricks. Why not let Gluster do the heavy lifting? > > > > > On Wed, Oct 15, 2014 at 7:35 PM, SINCOCK John > <J.Sincock@xxxxxxxxx> wrote: > > > In a related question... it seems, if it is possible to add > filesystems already containing data, as new bricks, then it > should also be possible to: > > 1) create empty bricks > 2) add them to the gluster volume while they are empty > 3) rsync data directly onto the underlying empty bricks, > circumventing gluster, ie not through the gluster mountpoint > 4) somehow get gluster to recognise the data that has been > copied into the bricks? > > How would you go about getting gluster to see the data you've > rsynced directly in? > My concern would be that all the data rsynced directly onto > the bricks will just sit there, invisible to glusterfs. > > Thanks again for any info! > > > -----Original Message----- > From: Franco Broi [mailto:franco.broi@xxxxxxxxxx] > Sent: Thursday, 16 October 2014 10:06 AM > To: SINCOCK John > Cc: gluster-users@xxxxxxxxxxx > Subject: Re: Is it ok to add a new brick with > files already on it? > > > > I've never added a brick with existing files but I did start a > new Gluster volume on disks that already contained data and I > was able to access the files without problem. Of course the > files will be out of place but the first time you access them, > Gluster will add links to speed up future lookups. > > On Thu, 2014-10-16 at 09:57 +1030, SINCOCK John wrote: > > Hi Everyone, > > > > > > > > All the instructions I’ve been able to find on adding a > brick to a > > gluster, seem to assume the brick is empty when it’s added. > > > > > > > > So my question is, is it possible for a new brick, loaded up > with > > files, to be added to a gluster (and for all the files > already on that > > brick, to be indexed and added into the gluster). Apologies > if the > > question is answered elsewhere, but I couldn’t find anyone > addressing > > this specific question, and certainty helps when you’re > dealing with > > 10’s of terabytes of data... ;-) > > > > > > > > Thanks in advance for any info or tips! > > > > > > > > > > _______________________________________________ > > Gluster-users mailing list > > Gluster-users@xxxxxxxxxxx > > > http://supercolony.gluster.org/mailman/listinfo/gluster-users > > > _______________________________________________ > Gluster-users mailing list > Gluster-users@xxxxxxxxxxx > http://supercolony.gluster.org/mailman/listinfo/gluster-users > > > > > > > _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://supercolony.gluster.org/mailman/listinfo/gluster-users
