Hi Yehuda, That¹s great. Is that backward compatiable with the previous configuration settings? That is to set rgw_max_chunk_size to 512 KB first and put some objects in size between 50 KB - 10 MB, and then set rgw_max_chunk_size to 1 MB, radosgw can read out the previously put objects. Thanks, Ray On 3/4/14, 9:01 AM, "Yehuda Sadeh" <yehuda@xxxxxxxxxxx> wrote: >Created issue #7589, and pushed a wip-7589 that addresses the issue. I >just ran some basic tests, so it should be taken with that in mind. > >Yehuda > >On Mon, Mar 3, 2014 at 5:31 AM, Ray Lv <raylv@xxxxxxxxxxxxx> wrote: >> Hi Yehuda, >> >> Thanks for your answers. >> >> The situation behind these questions is we found that an application >>data >> load is > 512KB and <4MB (maybe < 1 MB more precisely). The GET request >> usually requires 2 rados read ops. If there is a way we can configure >>the >> RGW_MAX_CHUNK_SIZE to a number between 512KB and 1MB, only 1 read op >>will >> be needed. So we can get total latency improved much even the first byte >> latency increased very little. >> >> Thanks, >> Ray >> >> On 2/21/14, 4:46 AM, "Yehuda Sadeh" <yehuda@xxxxxxxxxxx> wrote: >> >>>Apparently I missed including the mailing list in the response. >>> >>>---------- Forwarded message ---------- >>>From: Yehuda Sadeh <yehuda@xxxxxxxxxxx> >>>Date: Thu, Feb 20, 2014 at 9:04 AM >>>Subject: Re: [rgw] increase the first chunk size >>>To: Ray Lv <raylv@xxxxxxxxxxxxx> >>> >>> >>>On Thu, Feb 20, 2014 at 5:08 AM, Ray Lv <raylv@xxxxxxxxxxxxx> wrote: >>>> Hi, >>>> >>>> Currently, the first chunk size of a radosgw object is 512KB. >>>> >>>> Here is a case that most of data workload gets to radosgw is ~ 4MB. >>>>With the >>>> current first chunk size, each radosgw object is stripped to two >>>>chunks >>>> (512K + 3.5M). And we're using several large disks on each host with >>>>40TB >>>> capacity. So there will be 10 millions of files on each host. If the >>>>first >>>> chunk size is increased to 4MB, the number of files on each host will >>>>be >>>> reduced by 50%. It will be benifitial to performance of read because >>>>of >>>> reduced dcache and inode cache footprint in main memory (in other >>>>words, >>>> increased cache hit ratio). >>>> >>>> The questions are: >>>> >>>> What's the rationale behind for the current first chunk size? >>> >>>The head side conforms to the read chunk side. The gateway reads in >>>512k chunks. This is the basic read unit, and when accessing an object >>>we access the head only once and read the entire data + all its >>>attributes in one compound rados operation. This is done to ensure >>>atomicity. We don't have any knowledge as to whether the operation is >>>deemed to fail later on (e.g., insufficient permissions), so it's >>>going to read it anyway. >>> >>>> Are there any side effects if it is increased to 4MB? >>> >>>Will require reading the head in a single operation, which can be a >>>problem with regard to concurrency, add latency to all operations >>>(will take more time to stream data back to the client). Unauthorized >>>requests will use more resources. >>> >>> >>>Yehuda _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |