First you need to identify what kind of workload you will have.
Some optimizations for one workload can prevent better performance in another type.
If you plan to use the volume for small files , this document is a good start https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.5/html/administration_guide/small_file_performance_enhancements
Replica 2 volumes are prone to split brain and it's always recommended to have a replica 3 or an arbiter .
As a best practice - always test the volume with the application that will use it as synthetic benchmarks are just synthetics.
I always start performance review from the bottom (infrastructure) and end on application level.
You can start with https://access.redhat.com/documentation/en-us/red_hat_gluster_storage/3.5/html/administration_guide/chap-configuring_red_hat_storage_for_enhancing_performance as the storage is one of the most important parts.
What kind of HW raid , how many disks, stripe size and stripe width that you used in LVM?
Do you use thin LVM ?
How did you create your XFS (isize is critical) ?
Have you used gluster's tuned profile ?
Jumbo frames ?
Then you will need to optimize the volume for small files (see the link above).
Does you app allow you to use libgfapi ? Based on my observations in the oVirt list, libgfapi it used to provide some performance benefits compared to fuse .
Also, if you work with very small files, it would make sense to combine them in some container (like in VM disks).
Keep in mind that GlusterFS performance scales with the size of the cluster and the number of clients. For ultra high performance for a few clients -> there are other options.
Best Regards,
Strahil Nikolov
On Wed, Nov 9, 2022 at 12:05, beer Ll<llcfhllml@xxxxxxxxx> wrote:________HiI have 2 gluster server (backup1 , backup2) connected with 10Gbit linkglusterfs version 10.3-1 server and clientEach server is with 44T disk raid 6 with 1 partition used for the brick/dev/VG_BT_BRICK1/LV_BT_BRICK1 [ 43.66 TiB]/dev/mapper/VG_BT_BRICK1-LV_BT_BRICK1 on /mnt/glusterfs type xfs (rw,noatime,nouuid,attr2,inode64,sunit=256,swidth=1536,noquota)I create a replica volume named backupVolume Name: backup
Type: Replicate
Number of Bricks: 1 x 2 = 2
Transport-type: tcp
Bricks:
Brick1: backup-1:/mnt/glusterfs/brick1
Brick2: backup-2:/mnt/glusterfs/brick1
Options Reconfigured:
cluster.granular-entry-heal: on
storage.fips-mode-rchecksum: on
transport.address-family: inet
nfs.disable: on
performance.client-io-threads: off
cluster.enable-shared-storage: enableThe volume backup is mounted with gluster client on /mnt/sharebackup-1:/backup on /mnt/share type fuse.glusterfs (rw,relatime,user_id=0,group_id=0,default_permissions,allow_other,max_read=131072)Test with smallfile utilityXFSwith filesystem xfs on /mnt/glusterfstotal threads = 8
total files = 800
100.00% of requested files processed, warning threshold is 70.00%
elapsed time = 0.009
files/sec = 120927.642211GLUSTERFS CLIENTwith glusterfs on /mnt/sharetotal threads = 8
total files = 800
100.00% of requested files processed, warning threshold is 70.00%
elapsed time = 3.014
files/sec = 284.975861How is it possible to increase the performance in glusterfs volume ?How is best practice of brick size and replica management ?Is better 1 big brick per server or more small bricks distributed ?Many Thanks
Community Meeting Calendar:
Schedule -
Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC
Bridge: https://meet.google.com/cpu-eiue-hvk
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users
________ Community Meeting Calendar: Schedule - Every 2nd and 4th Tuesday at 14:30 IST / 09:00 UTC Bridge: https://meet.google.com/cpu-eiue-hvk Gluster-users mailing list Gluster-users@xxxxxxxxxxx https://lists.gluster.org/mailman/listinfo/gluster-users
