Re: high load when copy directory with many files

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



hi,
we have isolated the problem (meanwhile some hardware upgrade and code optimization helped to limit the problem). it happens when many request (HTTP over apache) comes to a non existent file. With 30 concurrent request to the same non existing file cause the load go high without limit.
Same requests on existing files works fine.
I have tried to simulate che apache access to file excluding apache with repeated command on files with the same parallelism (30):
- with ls works fine, file exists or not
- with stat works fine, file exists or not
- with xattr load go up, file exists or not

thank you


Il 05/10/2020 19.45, Marco Lerda - FOREACH S.R.L. ha scritto:
hi,
we use glusterfs on a php application that have many small php files images etc...
We use glusterfs in replication mode.
We have 2 nodes connected in fiber with 100MBps and less than 1 ms latency. We have also an arbiter on slower network (but the issue is there also without the arbiter). When we copy a directory (cp command) with many files, cpu usage and load explode raplidly,
our application become inaccessible until the copy ends.

I wonder if is that normal or we have done something wrong.
I know that glusterfs is not indicated with many small files, and I know that it slow down, but I want to avoid that a simple copy of a directory will put down out application.

Any suggestion?

Thanks a lot



________



Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://bluejeans.com/441850968

Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users

--

------------------------------------------------------
Marco Lerda
FOREACH S.R.L.
Via Laghi di Avigliana 115, 12022 - Busca (CN)
Telefono: 0171-1984102
Centralino/Fax: 0171-1984100
Email:  marco.lerda@xxxxxxxxxx
Web: http://www.foreach.it

________



Community Meeting Calendar:

Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users



[Index of Archives]     [Gluster Development]     [Linux Filesytems Development]     [Linux ARM Kernel]     [Linux ARM]     [Linux Omap]     [Fedora ARM]     [IETF Annouce]     [Bugtraq]     [Linux OMAP]     [Linux MIPS]     [eCos]     [Asterisk Internet PBX]     [Linux API]

  Powered by Linux