Hi, cephers:
I'm using ceph Luminous. Recently, one of my colleagues put his
gcc-tools , a whole binary folder with corresponding lib-files, into
cephfs. However, when running arm-gcc command to compile files,
arm-gcc always complains about "cc1plus" not found. If the whole
binary folder is located under a local filesystem, everything is ok.
Here is some pieces of log with strace when run arm-gcc under cephfs:
open("/tmp/ccIlAAE5.s", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
close(3) = 0
stat64("/mnt/cephfs/CentOS-6.6/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/../libexec/gcc/arm-linux-gnueabihf/4.9.2/cc1plus",
{st_mode=S_IFREG|0755, st_size=16224744, ...}) = 0
stat64("/mnt/cephfs/CentOS-6.6/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/../libexec/gcc/arm-linux-gnueabihf/cc1plus",
0xfffe9014) = -1 ENOENT (No such file or directory)
stat64("/mnt/cephfs/CentOS-6.6/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/../libexec/gcc/cc1plus",
0xfffe9014) = -1 ENOENT (No such file or directory)
stat64("/mnt/cephfs/CentOS-6.6/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/../lib/gcc/arm-linux-gnueabihf/4.9.2/../../../../arm-linux-gnueabihf/bin/arm-linux-gnueabihf/4.9.2/cc1plus",
0xfffe9014) = -1 ENOENT (No such file or directory)
stat64("/mnt/cephfs/CentOS-6.6/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/../lib/gcc/arm-linux-gnueabihf/4.9.2/../../../../arm-linux-gnueabihf/bin/arm-linux-gnueabihf/cc1plus",
0xfffe9014) = -1 ENOENT (No such file or directory)
stat64("/mnt/cephfs/CentOS-6.6/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/../lib/gcc/arm-linux-gnueabihf/4.9.2/../../../../arm-linux-gnueabihf/bin/cc1plus",
0xfffe9014) = -1 ENOENT (No such file or directory)
vfork(arm-linux-gnueabihf-g++: error trying to exec 'cc1plus': execvp:
No such file or directory
) = 8885
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=8885,
si_status=255, si_utime=0, si_stime=0} ---
waitpid(8885, [{WIFEXITED(s) && WEXITSTATUS(s) == 255}], 0) = 8885
stat64("/tmp/ccIlAAE5.s", {st_mode=S_IFREG|0600, st_size=0, ...}) = 0
unlink("/tmp/ccIlAAE5.s") = 0
exit_group(1) = ?
It shows arm-gcc has indeed found cc1plus, but somehow it continues to
search other directories.
Below gives log for running arm-gcc under local filesystem:
open("/tmp/ccU6Mkbg.s", O_RDWR|O_CREAT|O_EXCL, 0600) = 3
close(3) = 0
stat64("/mnt/local/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/../libexec/gcc/arm-linux-gnueabihf/4.9.2/cc1plus",
{st_mode=S_IFREG|0755, st_size=16224744, ...}) = 0
access("/mnt/local/gcc-linaro-arm-linux-gnueabihf-4.9-2014.09_linux/bin/../libexec/gcc/arm-linux-gnueabihf/4.9.2/cc1plus",
X_OK) = 0
vfork() = 8866
waitpid(8866, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0) = 8866
--- SIGCHLD {si_signo=SIGCHLD, si_code=CLD_EXITED, si_pid=8866,
si_status=0, si_utime=16, si_stime=5} ---
The main difference is that when using a local filesystem, arm-gcc
will not continue to look up in other directories after it has found
cc1plus. I'm really confused by this behavior. MDS's log just showed
it received some lookup requests and responded each request correctly.
The /mnt/local/CentOS-6.6 is copied from /mnt/cephfs/CentOS-6.6 with
`cp -rp` command, so symbol-link or file-mode should keep the same. I
tried gcc in this gcc-tool directory, and it worked well under cephfs.
Probably there is a bug in arm-gcc, but I can't imagine what code will
lead to such strange behavior.
Could anyone give some tips on this? Thanks!
