Increasing that shouldn't be problematic. The real issue is when decreasing it. First, you'd be throwing object atomicity out the window so with concurrent readers and writers to the same object you might end up having a reader getting inconsistent data. And second, it hasn't really been tested. Yehuda On Tue, Mar 4, 2014 at 6:55 AM, Ray Lv <raylv@xxxxxxxxxxxxx> wrote: > Hi Yehuda, > > That¹s great. Is that backward compatiable with the previous configuration > settings? That is to set rgw_max_chunk_size to 512 KB first and put some > objects in size between 50 KB - 10 MB, and then set rgw_max_chunk_size to > 1 MB, radosgw can read out the previously put objects. > > Thanks, > Ray > > > On 3/4/14, 9:01 AM, "Yehuda Sadeh" <yehuda@xxxxxxxxxxx> wrote: > >>Created issue #7589, and pushed a wip-7589 that addresses the issue. I >>just ran some basic tests, so it should be taken with that in mind. >> >>Yehuda >> >>On Mon, Mar 3, 2014 at 5:31 AM, Ray Lv <raylv@xxxxxxxxxxxxx> wrote: >>> Hi Yehuda, >>> >>> Thanks for your answers. >>> >>> The situation behind these questions is we found that an application >>>data >>> load is > 512KB and <4MB (maybe < 1 MB more precisely). The GET request >>> usually requires 2 rados read ops. If there is a way we can configure >>>the >>> RGW_MAX_CHUNK_SIZE to a number between 512KB and 1MB, only 1 read op >>>will >>> be needed. So we can get total latency improved much even the first byte >>> latency increased very little. >>> >>> Thanks, >>> Ray >>> >>> On 2/21/14, 4:46 AM, "Yehuda Sadeh" <yehuda@xxxxxxxxxxx> wrote: >>> >>>>Apparently I missed including the mailing list in the response. >>>> >>>>---------- Forwarded message ---------- >>>>From: Yehuda Sadeh <yehuda@xxxxxxxxxxx> >>>>Date: Thu, Feb 20, 2014 at 9:04 AM >>>>Subject: Re: [rgw] increase the first chunk size >>>>To: Ray Lv <raylv@xxxxxxxxxxxxx> >>>> >>>> >>>>On Thu, Feb 20, 2014 at 5:08 AM, Ray Lv <raylv@xxxxxxxxxxxxx> wrote: >>>>> Hi, >>>>> >>>>> Currently, the first chunk size of a radosgw object is 512KB. >>>>> >>>>> Here is a case that most of data workload gets to radosgw is ~ 4MB. >>>>>With the >>>>> current first chunk size, each radosgw object is stripped to two >>>>>chunks >>>>> (512K + 3.5M). And we're using several large disks on each host with >>>>>40TB >>>>> capacity. So there will be 10 millions of files on each host. If the >>>>>first >>>>> chunk size is increased to 4MB, the number of files on each host will >>>>>be >>>>> reduced by 50%. It will be benifitial to performance of read because >>>>>of >>>>> reduced dcache and inode cache footprint in main memory (in other >>>>>words, >>>>> increased cache hit ratio). >>>>> >>>>> The questions are: >>>>> >>>>> What's the rationale behind for the current first chunk size? >>>> >>>>The head side conforms to the read chunk side. The gateway reads in >>>>512k chunks. This is the basic read unit, and when accessing an object >>>>we access the head only once and read the entire data + all its >>>>attributes in one compound rados operation. This is done to ensure >>>>atomicity. We don't have any knowledge as to whether the operation is >>>>deemed to fail later on (e.g., insufficient permissions), so it's >>>>going to read it anyway. >>>> >>>>> Are there any side effects if it is increased to 4MB? >>>> >>>>Will require reading the head in a single operation, which can be a >>>>problem with regard to concurrency, add latency to all operations >>>>(will take more time to stream data back to the client). Unauthorized >>>>requests will use more resources. >>>> >>>> >>>>Yehuda > _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |