On Thu, Sep 24, 2015 at 7:26 PM, Gregory Farnum <gfarnum@xxxxxxxxxx> wrote: > On Thu, Sep 24, 2015 at 8:04 AM, Sage Weil <sweil@xxxxxxxxxx> wrote: >> On Thu, 24 Sep 2015, Robert LeBlanc wrote: >>> -----BEGIN PGP SIGNED MESSAGE----- >>> Hash: SHA256 >>> >>> >>> On Thu, Sep 24, 2015 at 6:30 AM, Sage Weil wrote: >>> > Xuan Liu recently pointed out that there is a problem with our handling >>> > for full clusters/pools: we don't allow any writes when full, >>> > including delete operations. >>> > >>> > While fixing a separate full issue I ended up making several fixes and >>> > cleanups in the full handling code in >>> > >>> > https://github.com/ceph/ceph/pull/6052 >>> > >>> > The interesting part of that is that we will allow a write as long as it >>> > doesn't increase the overall utilizate of bytes or objects (according to >>> > the pg stats we're maintaining). That will include remove ops, of cours, >>> > but will also allow overwrites while full, which seems fair. >>> >>> What about overwrites on a COW FS, won't that still increase used >>> space? Maybe if it is a COW FS, don't allow overwrites? >> >> Yeah, we could strengthen (optionally?) the check so that only operations >> that result in a net decrease are allowed.. > > It's not just COW filesystems, anything that modifies > leveldb/rocksdb/whatever in any way will also increase the space used > — including regular object deletes which additionally get added to the > PG log, although *hopefully* that's not a problem since we have our > extra buffers to handle this sort of thing. While right now we might > have some hope of being able to tag ops as "net deletes" or "net > adds", I don't see that happening once we have widespread third-party > object classes or that Lua work gets in or something... > > So, I'd be really leery of trying to do anything more advanced than > letting clients execute delete operations, and letting privileged > clients keep doing real work. (Or maybe restricting it entirely to the > second half there.) > > That latter switch already exists, by the way, although I don't think > it's actually enforced via cephx caps (it should be) — the Objecter > has an honor_full_flag setting which the MDS sets to false. I don't > think the library interfaces are there to specify it per-op but IIRC > it is part of the data sent to OSDs so it wouldn't require a wire > protocol change. I don't think it's sent with the ops, the OSDs simply check if the requester's entity type is MDS. John > -Greg > -- > To unsubscribe from this list: send the line "unsubscribe ceph-devel" in > the body of a message to majordomo@xxxxxxxxxxxxxxx > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
