|
Hi Chris,
The osds are running on arm nodes. Every node has
a two core 1.5Ghz arm 32bit cpu and 2G ram and runs 2 osds.Hdd is
10TB and journal colocate with data on the same disk.
Drives are half full now,but the problem I
described also happened when the hdds are empty. Filesystem is ext4
because I have some problem to run xfs for now.
I am trying to better balance the pg distrubition
now to see if it can ease the high latency problem.
发件人：Chris Hoy Poy <ChrisH@xxxxxxxxxxxx>
发送时间：2018-03-10 09:44
主题：Re: [ceph-users] [jewel] High fs_apply_latency
osds
收件人："shadow_lin"<shadow_lin@xxxxxxx>,"ceph-users"<ceph-users@xxxxxxxxxxxxxx>
抄送：
Hi Shadowlin,
Can you describe your hardware ? Cpu/ram/harddrives involved etc
Also how your drives are set up?
How full are the drives? What filesystem is it?
Cheers
/chris
Sent from my SAMSUNG Galaxy S6 on
the Telstra Mobile Network
-------- Original message --------
From: shadow_lin <shadow_lin@xxxxxxx>
Date: 10/3/18 1:41 am (GMT+08:00)
To: ceph-users <ceph-users@xxxxxxxxxxxxxx>
Subject: [ceph-users] [jewel] High fs_apply_latency osds
Hi list,
During my write test,I find there are always
some of the osds have high fs_apply_latency（1k-5kms,2-8times more than
others）. At first I think it is caused by unbalanced pg distribution, but
after I reweight the osds the problem hasn't gone away.
I looked into the osds with high latency and
find one thing in common.There are alot of read activities on these
osds.Result of iostat shows there are 400-500 r/s and 2000-3000 rKB/s on these
high latnecy osds while the normal osds have around 100 r/s 300-400
rKB/s.
I tried to restart osd daemon with high
latency.It did became normal sometime but there will be another high latency
osds come out.And the new high latency osd has the same high read
activities.
What are these read activities for? Is there a
way to lower the latency?
Thanks
Message protected by MailGuard: e-mail anti-virus, anti-spam and content
filtering. http://www.mailguard.com.auReport
this message as spam
Message protected by MailGuard: e-mail anti-virus, anti-spam and content
filtering.
http://www.mailguard.com.au
|
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
