So where did the decoding process happen? Is it in cache or on the client side? (Only considering Read.) If it happened when copying from storage tier to cache, then it has to be an whole object (file), but if decoding can be happened on client side when the client has all needed chunks, it seems cache can hold partial chunks of the file? What i mean is that is it possible for cache to hold partial chunks of a file in Ceph? (assuming file A has 7 chunks in storage tier, to recover file A a client needs 4 chunks, will it be possible that 2 chunks of file A are copied to and stored in cache, when file A is requested, only another 2 chunks are needed from the storage tier? )
Thanks!
e-----Original Message-----
From: huang jun <hjwsm1989@xxxxxxxxx>
To: Yu Xiang <hellomorning@xxxxxxxxxxxxx>
Cc: ceph-users <ceph-users@xxxxxxxxxxxxxx>
Sent: Wed, Mar 30, 2016 9:04 pm
Subject: Re: chunk-based cache in ceph with erasure coded back-end storage
if your cache-mode is write-back, which will cache the read object in
cache tier.
you can try the read-proxy mode, which will not cache the object.
the read request send to primary OSD, and the primary osd collect the
shards from base tier(in you case, is erasure code pool),
you need to read at least k chunks to decode the object.
In current code, cache tier only store the whole object, not the shards.
2016-03-31 6:10 GMT+08:00 Yu Xiang <hellomorning@xxxxxxxxxxxxx>:
> Dear List,
> I am exploring in ceph caching tier recently, considering a cache-tier
> (replicated) and a back storage-tier (erasure-coded), so chunks are stored
> in the OSDs in the erasure-coded storage tier, when a file has been
> requested to read, usually, all chunks in the storage tier would be copied
> to the cache tier, replicated, and stored in the OSDs in caching pool, but i
> was wondering would it be possible that if only partial chunks of the
> requested file be copied to cache? or it has to be a complete file? for
> example, a file using (7,4) erasure code (4 original chunks, 3 encoded
> chunks), when read it might be 4 required chunks are copied to cache, and i
> was wondering if it's possible to copy only 2 out of 4 required chunks to
> cache, and the users getting the other 2 chunks elsewhere (or assuming the
> client already has 2 chunks, they only need another 2 from ceph)? can the
> cache store partial chunks of a file?
>
> Thanks in advance for any help!
>
> Best,
> Yu
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
--
thanks
huangjun
To: Yu Xiang <hellomorning@xxxxxxxxxxxxx>
Cc: ceph-users <ceph-users@xxxxxxxxxxxxxx>
Sent: Wed, Mar 30, 2016 9:04 pm
Subject: Re: chunk-based cache in ceph with erasure coded back-end storage
if your cache-mode is write-back, which will cache the read object in
cache tier.
you can try the read-proxy mode, which will not cache the object.
the read request send to primary OSD, and the primary osd collect the
shards from base tier(in you case, is erasure code pool),
you need to read at least k chunks to decode the object.
In current code, cache tier only store the whole object, not the shards.
2016-03-31 6:10 GMT+08:00 Yu Xiang <hellomorning@xxxxxxxxxxxxx>:
> Dear List,
> I am exploring in ceph caching tier recently, considering a cache-tier
> (replicated) and a back storage-tier (erasure-coded), so chunks are stored
> in the OSDs in the erasure-coded storage tier, when a file has been
> requested to read, usually, all chunks in the storage tier would be copied
> to the cache tier, replicated, and stored in the OSDs in caching pool, but i
> was wondering would it be possible that if only partial chunks of the
> requested file be copied to cache? or it has to be a complete file? for
> example, a file using (7,4) erasure code (4 original chunks, 3 encoded
> chunks), when read it might be 4 required chunks are copied to cache, and i
> was wondering if it's possible to copy only 2 out of 4 required chunks to
> cache, and the users getting the other 2 chunks elsewhere (or assuming the
> client already has 2 chunks, they only need another 2 from ceph)? can the
> cache store partial chunks of a file?
>
> Thanks in advance for any help!
>
> Best,
> Yu
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
--
thanks
huangjun
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
