hi,
Here is the comparison of command execution times of 'volume set' on v3.3.0 and master:
On master:
⚡ time gluster volume set r2 performance.read-ahead off
volume set: success
real 0m1.250s
user 0m0.059s
sys 0m0.019s
On v3.3.0:
⚡ time gluster volume set r2 performance.read-ahead off
Set volume successful
real 0m0.081s
user 0m0.051s
sys 0m0.016s
The contributing factors to the delay are opening files in gluster-store with O_SYNC and fsync of the directory fd. With the following changes, the execution times came down to:
⚡ time gluster volume set r2 read-ahead off
volume set: success
real 0m0.130s
user 0m0.063s
sys 0m0.019s
⚡ g d
diff --git a/xlators/mgmt/glusterd/src/glusterd-store.c b/xlators/mgmt/glusterd/src/glusterd-store.c
index 863b70c..a225dfd 100644
--- a/xlators/mgmt/glusterd/src/glusterd-store.c
+++ b/xlators/mgmt/glusterd/src/glusterd-store.c
@@ -89,7 +89,7 @@ glusterd_store_mkstemp (glusterd_store_handle_t *shandle)
GF_ASSERT (shandle->path);
snprintf (tmppath, sizeof (tmppath), "%s.tmp", shandle->path);
- fd = open (tmppath, O_RDWR | O_CREAT | O_TRUNC | O_SYNC, 0600);
+ fd = open (tmppath, O_RDWR | O_CREAT | O_TRUNC, 0600);
if (fd <= 0) {
gf_log (THIS->name, GF_LOG_ERROR, "Failed to open %s, "
"error: %s", tmppath, strerror (errno));
@@ -121,12 +121,12 @@ glusterd_store_sync_direntry (char *path)
goto out;
}
- ret = fsync (dirfd);
- if (ret) {
- gf_log (this->name, GF_LOG_ERROR, "Failed to fsync %s, due to "
- "%s", pdir, strerror (errno));
- goto out;
- }
+ //ret = fsync (dirfd);
+ //if (ret) {
+ // gf_log (this->name, GF_LOG_ERROR, "Failed to fsync %s, due to "
+ // "%s", pdir, strerror (errno));
+ // goto out;
+ //}
ret = 0;
out:
run-tests.sh executed in 20 minutes with this change as opposed to 30 minutes without this change on my machine.
If we remove the O_SYNC flag alone and keep fsync of dir as is, here is the execution time:
⚡ time gluster volume set r2 read-ahead off
volume set: success
real 0m0.653s
user 0m0.054s
sys 0m0.015s
Do you feel we need to change anything based on these execution time differences?
It will save quite a bit of time at least for gerrit builds if there is a way to turn them off.
Let me know your views.
Pranith.
