Al 31/05/13 11:48, En/na Stefano Sinigardi ha escrit: > Dear Xavier, > I realized that the volume was not build properly when doing the first > analyses suggested by Davide, but I'm sure that this is not the > problem and so I quickly dismissed it. Also, we need a replica but not > so strictly, maybe in the future with the next volume I'll build it > properly. Anyway, yes, the volume got birth on "pedrillo" with a > replica-2 and the next day was expanded on "osmino", again with > replica-2, just by adding bricks and doing a rebalance, that was just > tried. I'm saying "tried" because it got "stuck", consuming a lot of > RAM (almost all, 16 GB), and it was counting million of files that I > think don't even exist on the volume, so I stopped it. Do you think > that it might be worth restarting? > The brick logs are there, just before your nice reply asking for them. > They're all the same, except for the order of the connection, all seem > very good. Sorry I don't know what I saw. Yes, the logs are there... :p > Yes, you're right, many files are missing through the mounpoint, I > just choose one. But then again, what do you think about the fact that > ls hides them but calling them from the mount point are not counted as > "file not found"? "leggi_particelle" was working calling it from the > bricks and from the mountpoint, even if ls didn't show it in the > mountpoint... It can happen. When you list a directory, a file may not be shown if it has some problem. However it can be reached if you access it directly (I have seen this before). > And here it is all the other info that you were asking: first of all a > collection of ls of the folder on all the bricks of the volume (10) > /storage/1/data/stefano/leggi_particelle: > total 20 > drwxr-xr-x 3 stefano user 4096 May 22 18:24 ./ > drwxr-xr-x 14 stefano user 4096 May 28 04:58 ../ > -rwxr-xr-x 2 stefano user 286 Feb 25 17:24 Espec.plt* > lrwxrwxrwx 2 stefano user 53 Feb 13 11:30 parametri.cpp -> > /tamino/stefano/codice/leggi_particelle/parametri.cpp > drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/ > /storage/2/data/stefano/leggi_particelle: > total 20 > drwxr-xr-x 3 stefano user 4096 May 22 18:24 ./ > drwxr-xr-x 14 stefano user 4096 May 28 04:58 ../ > -rwxr-xr-x 2 stefano user 286 Feb 25 17:24 Espec.plt* > lrwxrwxrwx 2 stefano user 53 Feb 13 11:30 parametri.cpp -> > /tamino/stefano/codice/leggi_particelle/parametri.cpp > drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/ > /storage/5/data/stefano/leggi_particelle: > total 892 > drwxr-xr-x 3 stefano user 4096 May 24 17:16 ./ > drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../ > lrwxrwxrwx 2 stefano user 50 Apr 11 19:20 filtro.cpp -> > /tamino/stefano/codice/leggi_particelle/filtro.cpp > lrwxrwxrwx 2 stefano user 70 Apr 11 19:20 > leggi_binario_ALaDyn_fortran.h -> > /tamino/stefano/codice/leggi_particelle/leggi_binario_ALaDyn_fortran.h > -rwxr-xr-x 2 stefano user 705045 May 22 18:24 leggi_particelle* > -rwxr-xr-x 2 stefano user 61883 Dec 16 17:20 leggi_particelle.old01* > -rwxr-xr-x 2 stefano user 106014 Apr 11 19:20 leggi_particelle.old03* > ---------T 2 root root 0 May 24 17:16 parametri.cpp > drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/ > /storage/6/data/stefano/leggi_particelle: > total 892 > drwxr-xr-x 3 stefano user 4096 May 24 17:16 ./ > drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../ > lrwxrwxrwx 2 stefano user 50 Apr 11 19:20 filtro.cpp -> > /tamino/stefano/codice/leggi_particelle/filtro.cpp > lrwxrwxrwx 2 stefano user 70 Apr 11 19:20 > leggi_binario_ALaDyn_fortran.h -> > /tamino/stefano/codice/leggi_particelle/leggi_binario_ALaDyn_fortran.h > -rwxr-xr-x 2 stefano user 705045 May 22 18:24 leggi_particelle* > -rwxr-xr-x 2 stefano user 61883 Dec 16 17:20 leggi_particelle.old01* > -rwxr-xr-x 2 stefano user 106014 Apr 11 19:20 leggi_particelle.old03* > ---------T 2 root root 0 May 24 17:16 parametri.cpp > drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/ > /storage/arc1/data/stefano/leggi_particelle: > total 144 > drwxr-xr-x 3 stefano user 4096 May 22 18:24 ./ > drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../ > lrwxrwxrwx 2 stefano user 53 Feb 22 19:40 binnaggio.cpp -> > /tamino/stefano/codice/leggi_particelle/binnaggio.cpp > -rwxr-xr-x 2 stefano user 350 Feb 25 17:24 Etheta.plt* > lrwxrwxrwx 2 stefano user 72 Mar 22 2012 > leggi_binario_ALaDyn_fortran.cpp -> > /tamino/stefano/codice/leggi_particelle/leggi_binario_ALaDyn_fortran.cpp > lrwxrwxrwx 2 stefano user 55 Mar 22 2012 leggi_campi.cpp -> > /tamino/stefano/codice/leggi_particelle/leggi_campi.cpp > lrwxrwxrwx 2 stefano user 60 Apr 11 19:20 leggi_particelle.cpp -> > /tamino/stefano/codice/leggi_particelle/leggi_particelle.cpp > -rwxr-xr-x 2 stefano user 97536 Mar 25 12:46 leggi_particelle.old02* > -rwxr-xr-x 2 stefano user 923 May 12 12:42 plot_den.plt* > lrwxrwxrwx 2 stefano user 54 Mar 22 2012 swap_tools.cpp -> > /tamino/stefano/codice/leggi_particelle/swap_tools.cpp > drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/ > -rwxr-xr-x 2 stefano user 309 Feb 25 17:24 xpx.plt* > /storage/arc2/data/stefano/leggi_particelle: > total 144 > drwxr-xr-x 3 stefano user 4096 May 22 18:24 ./ > drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../ > lrwxrwxrwx 2 stefano user 53 Feb 22 19:40 binnaggio.cpp -> > /tamino/stefano/codice/leggi_particelle/binnaggio.cpp > -rwxr-xr-x 2 stefano user 350 Feb 25 17:24 Etheta.plt* > lrwxrwxrwx 2 stefano user 72 Mar 22 2012 > leggi_binario_ALaDyn_fortran.cpp -> > /tamino/stefano/codice/leggi_particelle/leggi_binario_ALaDyn_fortran.cpp > lrwxrwxrwx 2 stefano user 55 Mar 22 2012 leggi_campi.cpp -> > /tamino/stefano/codice/leggi_particelle/leggi_campi.cpp > lrwxrwxrwx 2 stefano user 60 Apr 11 19:20 leggi_particelle.cpp -> > /tamino/stefano/codice/leggi_particelle/leggi_particelle.cpp > -rwxr-xr-x 2 stefano user 97536 Mar 25 12:46 leggi_particelle.old02* > -rwxr-xr-x 2 stefano user 923 May 12 12:42 plot_den.plt* > lrwxrwxrwx 2 stefano user 54 Mar 22 2012 swap_tools.cpp -> > /tamino/stefano/codice/leggi_particelle/swap_tools.cpp > drwxr-xr-x 3 stefano user 4096 Apr 11 19:19 test/ > -rwxr-xr-x 2 stefano user 309 Feb 25 17:24 xpx.plt* > /storageOsmino/1/data/stefano/leggi_particelle: > total 24 > drwxr-xr-x 3 stefano user 4096 May 29 05:00 ./ > drwxr-xr-x 14 stefano user 4096 May 28 04:55 ../ > ---------T 2 root root 0 May 29 05:00 leggi_particelle > drwxr-xr-x 3 stefano user 4096 May 24 17:16 test/ > /storageOsmino/2/data/stefano/leggi_particelle: > total 24 > drwxr-xr-x 3 stefano user 4096 May 29 05:00 ./ > drwxr-xr-x 14 stefano user 4096 May 28 04:55 ../ > ---------T 2 root root 0 May 29 05:00 leggi_particelle > drwxr-xr-x 3 stefano user 4096 May 24 17:16 test/ > /storageOsmino/4/data/stefano/leggi_particelle: > total 16 > drwxr-xr-x 3 stefano user 4096 May 24 17:16 ./ > drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../ > drwxr-xr-x 3 stefano user 4096 May 24 17:16 test/ > /storageOsmino/5/data/stefano/leggi_particelle: > total 16 > drwxr-xr-x 3 stefano user 4096 May 24 17:16 ./ > drwxr-xr-x 14 stefano user 4096 May 28 11:32 ../ > drwxr-xr-x 3 stefano user 4096 May 24 17:16 test/ > ======================================= > ======================================= > and these are the attributes of the folders in every brick > # file: storage/1/data/stefano/leggi_particelle > trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105 > trusted.glusterfs.dht=0x00000001000000000000000033333332 > # file: storage/2/data/stefano/leggi_particelle > trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105 > trusted.glusterfs.dht=0x00000001000000000000000033333332 > # file: storage/5/data/stefano/leggi_particelle > trusted.afr.data-client-2=0x000000000000000000000000 > trusted.afr.data-client-3=0x000000000000000000000000 > trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105 > trusted.glusterfs.dht=0x00000001000000003333333366666665 > # file: storage/6/data/stefano/leggi_particelle > trusted.afr.data-client-2=0x000000000000000000000000 > trusted.afr.data-client-3=0x000000000000000000000000 > trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105 > trusted.glusterfs.dht=0x00000001000000003333333366666665 > # file: storage/arc1/data/stefano/leggi_particelle > trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105 > trusted.glusterfs.dht=0x0000000100000000ccccccccffffffff > # file: storage/arc2/data/stefano/leggi_particelle > trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105 > trusted.glusterfs.dht=0x0000000100000000ccccccccffffffff > # file: storageOsmino/1/data/stefano/leggi_particelle > trusted.afr.data-client-6=0x000000000000000000000000 > trusted.afr.data-client-7=0x000000000000000000000000 > trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105 > trusted.glusterfs.dht=0x00000001000000006666666699999998 > # file: storageOsmino/2/data/stefano/leggi_particelle > trusted.afr.data-client-6=0x000000000000000000000000 > trusted.afr.data-client-7=0x000000000000000000000000 > trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105 > trusted.glusterfs.dht=0x00000001000000006666666699999998 > # file: storageOsmino/4/data/stefano/leggi_particelle > trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105 > trusted.glusterfs.dht=0x000000010000000099999999cccccccb > # file: storageOsmino/5/data/stefano/leggi_particelle > trusted.gfid=0xb62a16f0bdb94e3f8563ccfb278c2105 > trusted.glusterfs.dht=0x000000010000000099999999cccccccb > =========================================== > Something is not consistent here and I can see it by myself... > I hope to be forgiven for all these details and spamming in the > mailing list. I'm just arrived and I'm already filling it up... Sorry. > But also thanks a lot for your help! > Stefano > I don't see anything incorrect here. It's very weird. Have you modified any file manually ? Can you look at .glusterfs/b6/2a/b62a16f0-bdb9-4e3f-8563-ccfb278c2105 on each brick ? is it a valid symbolic link pointing to data/stefano/leggi_particelle ? Check also .glusterfs/88/3c/883c343b-9366-478d-a660-843da8f6b87c. Also make sure if the file 'leggi_particelle' is present in any other brick. Isn't there anything different in the logs of first and second bricks ? I don't see what can be happening... Xavi
