A follow up. Gluster seems to do great with transfers of files (read / write), but runs into performance issues when performing computations on the files. Just like mysql server causing glusterfs & glusterfsd high cpu load when mounted on glusterfs, 5 concurrent md5sum operations on a 300MB file cause the same high cpu usage for glusterfs and glusterfsd (see below) PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 12868 root 15 0 22344 2964 904 S 32 0.1 1:06.01glusterfs 13622 root 17 0 3924 1624 512 R 30 0.1 0:00.91md5sum 12329 root 15 0 228m 81m 892 S 28 4.0 1:21.72glusterfsd 13621 root 16 0 3924 1620 512 D 27 0.1 0:01.02md5sum 13623 root 17 0 3924 1620 512 R 20 0.1 0:00.61md5sum 13620 root 16 0 3928 1628 512 R 19 0.1 0:00.89md5sum 13619 root 16 0 3928 1624 512 R 17 0.1 0:00.83md5sum 13618 root 16 0 3924 1620 512 R 17 0.1 0:00.98md5sum Also, you can see the mysql stats here -- mysql can only utilize 7% of cpu, but when not run on gluster, mysql utilization for SAME query is 50%. PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 12868 root 15 0 21316 1936 904 S 55 0.1 1:10.97glusterfs 12329 root 15 0 218m 63m 892 S 54 3.1 1:26.42glusterfsd 13534 mysql 15 0 44360 9340 3976 S 7 0.5 0:03.90mysqld Is this expected behavior? Is there a way to tune to reduce glusterfs/glusterfsd overhead? Thank you. On Fri, Feb 29, 2008 at 9:21 AM, billy cokalopolous < billy.cokalopolous@xxxxxxxxx> wrote: > I'm currently testing a 1 node architecture as I was having problems with > afr and multinode performance. > > write and read performance to glusterfs (single node) is now good, but > when running mysql on top of glusterfs, performance drops dramatically due > to high cpu usage. > > mysql is mounted to the glusterfs & glusterfsd is running on the same > node. > > When running a simple query accessing one table, the cpu load from > glusterfs / glusterfsd in top are relatively low. > > When running a 3 table join (18 files must be read, myi,myd,frm for each > table, 2 tables per virtual table, so 3*3*2), the glusterfs/glusterfsd cpu > load skyrockets stealing the cpu away from mysql processing. > > So this query on the local fs (ext3) takes 4 seconds, while the single > node glusterfs takes 22 seconds. > > I've adjusted iothreads, iocache, readahead, etc,etc. None of these > options make a difference in the cpu load. Tried using posix-locks and then > posix. > > I wonder if glusterfs and mysql are stepping on each other's toes to cause > this high load? Might the issue lie in tuning the file system & kernel > params? Or is this cpu usage expected load? > > Thanks. > >
