S3website range requests - possible issue

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]



In our deployment we are using the mix of s3 and s3website RGW. I’ve noticed strange behaviour when sending range requests to the s3website RGWs that I’m not able to replicate on the s3 ones.

I’ve created a simple wrk LUA script to test sending range requests on tiny ranges so the issue is easily seen.

When sending these requests against s3 RGW I can see that the amount of data read from Ceph is ± equivalent to what the RGW sends to the client. This change very dramatically when I’m doing the same test against s3website RGW. The read from Ceph is huge (3Gb/s compared to ~22Mb/s on s3 RGW) I seems to me like the RGW is reading the whole files and then sending just the range which is different then what s3 does.

I do not understand why would s3website need to read that much from Ceph and I believe this is a bug - I was looking through the tracker and wasn’t able to find anything related to s3website and range requests.

Did anyone else noticed this issue?

You can replicate it by running this wrk command wrk -t56 -c500 -d5m http://${rgwipaddress}:8080/${bucket}/videos/ -s wrk-range-small.lua

wrk script

-- Initialize the pseudo random number generator
math.randomseed( os.time())
math.random(); math.random(); math.random()

i = 1

function request()
    if i == 8
        i = 1

    local nrangefrom = math.random()
    local nrangeto = math.random(100)
    local path = wrk.path
    url = path..i..".mp4"
    wrk.headers["Range"] = nrangefrom.."-"..nrangeto
    i = i+1
    return wrk.format(nil, url)

Kind regards,

ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]

  Powered by Linux