Hi,
Yes, you can use cgroup to improve speed limit for gluster operation.
To use the cgroup for any specific glusterd(selfheald etc) first you have to change the glusterd unit file and start the service (glusterd and volumes).
I would like to suggest before apply the step on large cluster please do run the same steps on test vm and
if it is working fine on test vm then apply same on your large cluster.
1) Add directive "Delegate=yes" in Service section in glusterd unit file as below
[Service]
Type=forking
PIDFile=/var/run/glusterd.pid
LimitNOFILE=65536
Delegate=yes
Environment="LOG_LEVEL=INFO"
EnvironmentFile=-${prefix}/etc/sysconfig/glusterd
ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid --log-level $LOG_LEVEL $GLUSTERD_OPTIONS
KillMode=process
SuccessExitStatus=15
2) stop gluster services (kill volume also)
3) do daemon reload systemctl daemon-reload
4) start glusterd service
5) Create cgroup in subsystems (cpu,cpuacct,memory etc) whatever you want to restrict for specific daemon
here i am trying to control CPU for selfheald so i have created mycgroup directory in that subsystem
mkdir -p /sys/fs/cgroup/cpu,cpuacct/system.slice/glusterd.service/mycgroup
6) Set quota for this cgroup as below
echo 25000 > /sys/fs/cgroup/cpu,cpuacct/system.slice/glusterd.service/mycgroup/cpu.cfs_quota_us
here the cpu.cfs_quota_us means total available run-time within a period (microsecond) and length
of period is 100 ms , here 25000 represent 25 ms, it means kernel will allocate cpu for these tasks
for 25ms in every 100 ms
7) Attach the daemon tasks for that you want to control CPU
here i am trying to control selfheal daemon , suppose pid of selfheald is 576, run below command to move
all selfheald tasks to mycgroup
for thid in `ps -T -p 576 | grep gluster | awk -F " " '{print $2}'`; do echo $thid > /sys/fs/cgroup/cpu,cpuacct/system.slice/glusterd.service/mycgroup/tasks ; done
8) After attached all selfheald tasks to mygcgroup to ensure task is successfully attached or not you can check
tasks file in glusterd.service/tasks in the same hierarchy, all tasks will be move from glusterd tasks to mycgroup
tasks.
9) Check top output, cpu usage for selfheald will be lower.Let me know if u face any problem to run the steps.
Regards
Mohit Agrawal
On Tue, Aug 1, 2017 at 4:19 PM, Ravishankar N <ravishankar@xxxxxxxxxx> wrote:
Adding Mohit who is experimenting with cgroups has found some way to restrict glustershd's CPU usage using cgroups. Mohit maybe you want to share the steps we need to follow to apply cgroups only to glustershd.
Thanks.
Ravi
On 08/01/2017 03:46 PM, Alexey Zakurin wrote:
Hi, community
I have a large distributed-replicated Glusterfs volume, that contains few hundreds VM's images. Between servers 20Gb/sec link.
When I start some operations like healing or removing, storage performance becomes too low for a few days and server load becomes like this:
13:06:32 up 13 days, 20:02, 3 users, load average: 43.62, 31.75, 23.53.
Is it possible to set limit on this operations? Actually, VM's on my cluster becomes offline, when I start healing, rebalance or removing brick.
Please, help.
_______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://lists.gluster.org/mailman/listinfo/gluster-users
