On Wed, Apr 27, 2016 at 12:25:39PM -0400, Joseph Fernandes wrote: > Discussion till now. > > ~Joe > > ----- Forwarded Message ----- > > From: "Joseph Fernandes" <josferna@xxxxxxxxxx> > > To: "Dan Lambright" <dlambrig@xxxxxxxxxx> > > Sent: Wednesday, April 27, 2016 9:24:57 PM > > Subject: Re: Compressing CTR DB > > > > answers inline > > > > ----- Original Message ----- > > > From: "Dan Lambright" <dlambrig@xxxxxxxxxx> > > > Sent: Wednesday, April 27, 2016 8:39:06 PM > > > Subject: Re: Compressing CTR DB > > > > > > To compress the uuid, is it a lossless compression algorithm? So we won't > > > lose any bits? > > > > > > > We dont compress using any compression algos. Instead of saving the GFID > > (which is a uuid) as printable string which takes 33-36 bytes > > will save it as a 16 byte integer/blob, how it is represented in-memory > > > > > If we do an upgrade and change the schema, could we delete the old db and > > > start a brand new one? An upgrade is a rare event? > > > > In a upgrade scenario we can do the following untill GFID to PATH Conversion > > comes. > > 1. Decommission the old DB and start using new DB > > 2. The new DB will be healed in two ways, > > a. Named lookup : Though the File heat table will be healed using any > > other operation, the file link table (the one with multiple hardlinks) > > will not be healed until and unless there is a nameslookup. > > b. The Background heal from old to new via a separate thread in the brick. > > YES there might be a performance hit, and this can be contained using > > throttling mechanism. > > > > Again the question of how often a user upgrades ? might be its a rare event, > > but stability shouldnt be affected. > > > > As discussed in the scrum lets speak to Aravinda and Shyam about this issue > > of GFID to PATH Conversion next week, there is a proposal, but nothing > > implemented and > > functional as we have in DB. But yes we need to move it out of the DB as its > > not why we got the DB. > > > > > > > > Agree we need a version # as part of the solution. > > > > YES we will have a version of schema in the DB itself. > > > > > > > > > > > ----- Joseph Fernandes <josferna@xxxxxxxxxx> wrote: > > > > Hi All, > > > > > > > > As I am working on shrinking the CTR DB Size, I came across few of the > > > > articles/blogs on this. Just wondering if shrinking is really needed? Do you have some statistics on CTR DB size from users? > > > > As predicted, saving the UUID as 16 byte rather than 36 byte text will > > > > give > > > > us atleast 46% reduction > > > > in disk and cache space. Plus The blog do suggest some performance > > > > gain(if > > > > we don't often convert UUID to String, whi). I expect that almost any database has a native UUID/GUID format? Would that not be the most natural to use? Thanks, Niels > > > > http://www.google.com/url?q=http%3A%2F%2Fwtanaka.com%2Fnode%2F8106&sa=D&sntz=1&usg=AFQjCNEZolVlLAW2OGxq96CFjfeY0mQC1A > > > > https://scion.duhs.duke.edu/vespa/project/wiki/DatabaseUuidEfficiency > > > > > > > > The changes in the current libgfdb code is at the sqlite level. But since > > > > there is a change in schema, we need to write > > > > db data migration scripts during upgrades (Similar to dual connection > > > > path). Speaking of which, we would need DB schema versions > > > > and need to have it stored in gluster (either on glusterd or db or > > > > namespace), As we will expect the schema to change as we fine > > > > tune our heat store. > > > > > > > > Regards, > > > > Joe > > > > > > > > > > > > > > > _______________________________________________ > Gluster-devel mailing list > Gluster-devel@xxxxxxxxxxx > http://www.gluster.org/mailman/listinfo/gluster-devel
Attachment:
signature.asc
Description: PGP signature
_______________________________________________ Gluster-devel mailing list Gluster-devel@xxxxxxxxxxx http://www.gluster.org/mailman/listinfo/gluster-devel