On Thu, Oct 18, 2018 at 2:55 PM Robin H. Johnson <robbat2@xxxxxxxxxx> wrote: > > On Tue, Oct 16, 2018 at 06:08:10PM -0700, Yehuda Sadeh-Weinraub wrote: > > Here are my current thoughts about tiering, and also specifically > > about cloud tiering. > > > > 1. Storage-classes > > > > Previously a placement target would be mapped into a set of rados > > pools (index, data, extra), whereas now placement targets will add > > storage classes (S3 uses these). Object placement will be defined by > > the placement target, and the storage class. > ... > > We should probably make it so that when head and tail are being placed > > on different placement targets, the head will not contain any data, > > other than the object’s metadata. > > This ties into work I was hoping to have time in Q4 to work on. > > Right now, with erasure-coded pools, access to the metadata is very > slow, esp. for metadata-write workloads. > > At the risk of having more objects, I was wondering about a full split > of: > (metadata) > (head) > (tails) > Each of which might be a different pool, depending on user workload. > > Example policy I had in mind: > 1. Metadata goes to replicated SSD pool. > 2. (N) 4KB Head goes to EC SSD pool. N should be configurable. > 3. Tails go to EC spinner pool. > > These would be pools/targets selected based on the storage class. > > Zero-byte files wind up (metadata)-only. > Tiny files implicitly wind up with just (metadata)(head). Large files > wind up spread over all 3. > What you're asking for is actually a split tail, right? The only thing special about the head is its metadata (that includes its manifest), so without it it's just data. Unless you were thinking of splitting the manifest out? At the moment all parts of the tail need to be located at the same placement target. I can see a couple of options here: 1. Extend the manifest by adding placement_target to each manifest part, and modify object construction so that it creates new parts when needed (and update iteration so that it doesn't assume a single placement target for tail). 2. Extend placement target config so that it can have a list of <offset, dest>. Then need to update the code that maps from rgw_obj + placement_target to rgw_raw_obj to also include offset (and max size). Yehuda
