Re: Uploading large files to swift interface on radosgw

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



On Tue, Sep 17, 2013 at 3:21 PM, Gerd Jakobovitsch <gerd@xxxxxxxxxxxxx> wrote:
> Hi all,
>
> I am testing a ceph environment installed in debian wheezy, and, when
> testing file upload of more than 1 GB, I am getting errors. For files larger
> than 5 GB, I get a "400 Bad Request   EntityTooLarge" response; looking at

The EntityTooLarge is expected, as there's a 5GB limit on objects.
Bigger objects need to be uploaded using the large object api.

> the radosgw server, I notice that only the apache process is consuming cpu
> time, and I only have traffic on the external interface used by apache.
> For files between 2 GB  and 5 GB, I get stuck for a very long time, and I
> see relatively high processing for both apache and radosgw. Finally, I get a
> response "500 Internal Server Error UnknownError". The object is created on
> rados, but is empty.
>
> I am wondering whether there are any configuration I should change on
> apache, fastcgi or rgw, or if there are hardware limitations.
>
> Apache and fastCGI where installed from the distro. My ceph configuration:

Are you by any chance using the fcgi module rather than the fastcgi
module? It had a problem with caching the entire object before sending
it to the backend, which would result in the same symptoms as you just
described.

Yehuda

> [global]
> mon_initial_members = spcsmp1, spcsmp2, spcsmp3
> mon_host = 10.17.0.2,10.17.0.3,10.17.0.4
> auth_cluster_required = cephx
> auth_service_required = cephx
> auth_client_required = cephx
> osd_journal_size = 1024
> filestore_xattr_use_omap = true
> public_network = 10.17.0.0/24
> cluster_network = 10.18.0.0/24
>
> [osd]
> osd_journal_size = 1024
>
> [client.radosgw.gateway]
> host = mss.mandic.com.br
> keyring = /etc/ceph/keyring.radosgw.gateway
> rgw_socket_path = /tmp/radosgw.sock
> log_file = /var/log/ceph/radosgw.log
> rgw_enable_ops_log = false
>
>
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux