|
>> Would it be 4 objects of 24M and 4 objects of 250KB? Or will the last
4 objects be artificially padded (with 0's) to meet the stripe_unit?
It will be 4 object of 24M + 1M stored on the 5th object
If you write 104M : 4 object of 24M + 8M stored on the 5th object
If you write 105M : 4 object of 24M + 8M stored on the 5th object + 1M on 6th object
Maged
On 2017-10-17 01:59, Christian Wuerdig wrote:
Maybe an additional example where the numbers don't line up all so nicely would be good as well. For example it's not immediately obvious to me what would happen with the stripe settings given by your example but you write 97M of data Would it be 4 objects of 24M and 4 objects of 250KB? Or will the last 4 objects be artificially padded (with 0's) to meet the stripe_unit? On Tue, Oct 17, 2017 at 12:35 PM, Alexander Kushnirenko < kushnirenko@xxxxxxxxx> wrote:
Hi, Gregory, Ian!
There is very little information on striper mode in Ceph documentation.
Could this explanation help?
The logic of striper mode is very much the same as in RAID-0. There are 3
parameters that drives it:
stripe_unit - the stripe size (default=4M)
stripe_count - how many objects to write in parallel (default=1)
object_size - when to stop increasing object size and create new objects.
(default =4M)
For example if you write 132M of data (132 consecutive pieces of data 1M
each) in striped mode with the following parameters:
stripe_unit = 8M
stripe_count = 4
object_size = 24M
Then 8 objects will be created - 4 objects with 24M size and 4 objects with
8M size.
Obj1=24M Obj2=24M Obj3=24M Obj4=24M
00 .. 07 08 .. 0f 10 .. 17 18 .. 1f <-- consecutive
1M pieces of data
20 .. 27 21 .. 2f 30 .. 37 38 .. 3f
40 .. 47 48 .. 4f 50 .. 57 58 .. 5f
Obj5= 8M Obj6= 8M Obj7= 8M Obj8= 8M
60 .. 67 68 .. 6f 70 .. 77 78 .. 7f
Alexander.
On Wed, Oct 11, 2017 at 3:19 PM, Alexander Kushnirenko
<kushnirenko@xxxxxxxxx> wrote:
Oh! I put a wrong link, sorry The picture which explains stripe_unit and
stripe count is here:
https://indico.cern.ch/event/330212/contributions/1718786/attachments/642384/883834/CephPluginForXroot.pdf
I tried to attach it in the mail, but it was blocked.
On Wed, Oct 11, 2017 at 3:16 PM, Alexander Kushnirenko
<kushnirenko@xxxxxxxxx> wrote:
Hi, Ian!
Thank you for your reference!
Could you comment on the following rule:
object_size = stripe_unit * stripe_count
Or it is not necessarily so?
I refer to page 8 in this report:
https://indico.cern.ch/event/531810/contributions/2298934/attachments/1358128/2053937/Ceph-Experience-at-RAL-final.pdf
Alexander.
On Wed, Oct 11, 2017 at 1:11 PM, <ian.johnson@xxxxxxxxxx> wrote:
Hi Gregory
You're right, when setting the object layout in libradosstriper, one
should set all three parameters (the number of stripes, the size of the
stripe unit, and the size of the striped object). The Ceph plugin for
GridFTP has an example of this at
https://github.com/stfc/gridFTPCephPlugin/blob/master/ceph_posix.cpp#L371
At RAL, we use the following values:
$STRIPER_NUM_STRIPES 1
$STRIPER_STRIPE_UNIT 8388608
$STRIPER_OBJECT_SIZE 67108864
Regards,
Ian Johnson MBCS
Data Services Group
Scientific Computing Department
Rutherford Appleton Laboratory
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxxhttp://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
|
