Re: Uploading large files to swift interface on radosgw

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Thank you very much, now it worked, with the value you suggested.

Regards.

On 09/19/2013 12:10 PM, Yehuda Sadeh wrote:
Now you're hitting issue #6336 (it's a regression in dumpling that
we'll fix soon). The current workaround is setting the following in
your osd:

osd max attr size = <large number here>

try a value of 10485760 (10M) which I think is large enough.

Yehuda



On Thu, Sep 19, 2013 at 7:30 AM, Gerd Jakobovitsch <gerd@xxxxxxxxxxxxx> wrote:
Hello Yehuda, thank you for your help.


On 09/17/2013 08:35 PM, Yehuda Sadeh wrote:
On Tue, Sep 17, 2013 at 3:21 PM, Gerd Jakobovitsch <gerd@xxxxxxxxxxxxx> wrote:
Hi all,

I am testing a ceph environment installed in debian wheezy, and, when
testing file upload of more than 1 GB, I am getting errors. For files larger
than 5 GB, I get a "400 Bad Request   EntityTooLarge" response; looking at
The EntityTooLarge is expected, as there's a 5GB limit on objects.
Bigger objects need to be uploaded using the large object api.

the radosgw server, I notice that only the apache process is consuming cpu
time, and I only have traffic on the external interface used by apache.
For files between 2 GB  and 5 GB, I get stuck for a very long time, and I
see relatively high processing for both apache and radosgw. Finally, I get a
response "500 Internal Server Error UnknownError". The object is created on
rados, but is empty.

I am wondering whether there are any configuration I should change on
apache, fastcgi or rgw, or if there are hardware limitations.

Apache and fastCGI where installed from the distro. My ceph configuration:
Are you by any chance using the fcgi module rather than the fastcgi
module? It had a problem with caching the entire object before sending
it to the backend, which would result in the same symptoms as you just
described.

Yehuda
Well, I followed the installation instructions, that explicitly refer to fastcgi. Now I disabled the cgid module and repeated the test: I got the same problem.

Apache and fastcgi versions:
apache2:
   Installed: 2.2.22-13
libapache2-mod-fastcgi:
   Installed: 2.4.7~0910052141-1

I enabled radosgw logging; please find annex the log file. There is a lot of information listed, but I couldn't figure out the problem.

Regards.



[global]
mon_initial_members = spcsmp1, spcsmp2, spcsmp3
mon_host = 10.17.0.2,10.17.0.3,10.17.0.4
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
osd_journal_size = 1024
filestore_xattr_use_omap = true
public_network = 10.17.0.0/24
cluster_network = 10.18.0.0/24

[osd]
osd_journal_size = 1024

[client.radosgw.gateway]
host = mss.mandic.com.br
keyring = /etc/ceph/keyring.radosgw.gateway
rgw_socket_path = /tmp/radosgw.sock
log_file = /var/log/ceph/radosgw.log
rgw_enable_ops_log = false



_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com



Attachment: smime.p7s
Description: S/MIME Cryptographic Signature

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux