Dear all,
I have a situation where "mkdir" on a client produces stale file handles.
This happened after upgrading from 3.12 to 6.5
I believe I found the reason for it:
6.5 (but not 3.12) checks if there is space left on the device before doing a "mkdir", but calculates the "fullness" in percent. In my situations I have bricks that seem 100% full although there is plenty space left on the device (several GBytes, see listing below). In this situation, a "mkdir" is not performed on bricks that are 100% full, but the "mkdir" succeeds from a user perspective. Then, doing a "ls" on the recently created directory leads to the message "stale file handle".
I believe the call sequence is more or less this:
server-rpc-fops.c:539:server_mkdir_cbk
server-rpc-fops.c:2666:server_mkdir_resume
server-rpc-fops.c:5242:server3_3_mkdir
posix-entry-ops.c:625:posix_mkdir
posix-helpers.c:2271
My questions are:
* is it meant to operate in this way?
* is there a built-in way to fix the inconsistent directories?
(I tried creating the missing directories on the bricks by hand, which seemed to fix the issue, but I'm not sure if this will introduce other problems.)
The obvious (good) fix would be to redistribute the data such that the 100% full bricks will have enough free space. However, if a user writes a really large file, the problem can re-occur any time...
best wishes,
Stefan
PS:
File system listing. Each file system is served as a brick, in a distribute-only system.
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/vgosb03pool06vd03-lvosb03pool06vd03 30T 27T 3.8T 88% /gl/lvosb03pool06vd03
/dev/mapper/vgosb03pool06vd02-lvosb03pool06vd02 30T 27T 3.8T 88% /gl/lvosb03pool06vd02
/dev/mapper/vgosb03pool06vd01-lvosb03pool06vd01 30T 27T 3.7T 88% /gl/lvosb03pool06vd01
/dev/mapper/vgosb03pool01vd01-lvosb03pool01vd01 30T 30T 7.8G 100% /gl/lvosb03pool01vd01
/dev/mapper/vgosb03pool01vd02-lvosb03pool01vd02 30T 30T 41G 100% /gl/lvosb03pool01vd02
/dev/mapper/vgosb03pool01vd03-lvosb03pool01vd03 30T 29T 1.5T 96% /gl/lvosb03pool01vd03
/dev/mapper/vgosb03pool01vd04-lvosb03pool01vd04 30T 30T 17G 100% /gl/lvosb03pool01vd04
/dev/mapper/vgosb03pool02vd01-lvosb03pool02vd01 30T 30T 57G 100% /gl/lvosb03pool02vd01
/dev/mapper/vgosb03pool02vd02-lvosb03pool02vd02 30T 30T 29G 100% /gl/lvosb03pool02vd02
/dev/mapper/vgosb03pool02vd03-lvosb03pool02vd03 30T 30T 26G 100% /gl/lvosb03pool02vd03
/dev/mapper/vgosb03pool02vd04-lvosb03pool02vd04 31T 31T 9.7G 100% /gl/lvosb03pool02vd04
/dev/mapper/vgosb03pool03vd01-lvosb03pool03vd01 30T 30T 93G 100% /gl/lvosb03pool03vd01
/dev/mapper/vgosb03pool03vd02-lvosb03pool03vd02 30T 30T 23G 100% /gl/lvosb03pool03vd02
/dev/mapper/vgosb03pool03vd03-lvosb03pool03vd03 30T 30T 163G 100% /gl/lvosb03pool03vd03
/dev/mapper/vgosb03pool03vd04-lvosb03pool03vd04 31T 30T 384G 99% /gl/lvosb03pool03vd04
/dev/mapper/vgosb03pool04vd01-lvosb03pool04vd01 30T 29T 1.1T 97% /gl/lvosb03pool04vd01
/dev/mapper/vgosb03pool04vd02-lvosb03pool04vd02 30T 27T 3.9T 88% /gl/lvosb03pool04vd02
/dev/mapper/vgosb03pool04vd03-lvosb03pool04vd03 30T 29T 1.9T 94% /gl/lvosb03pool04vd03
/dev/mapper/vgosb03pool04vd04-lvosb03pool04vd04 31T 29T 1.9T 94% /gl/lvosb03pool04vd04
/dev/mapper/vgosb03pool05vd01-lvosb03pool05vd01 30T 28T 2.3T 93% /gl/lvosb03pool05vd01
/dev/mapper/vgosb03pool05vd02-lvosb03pool05vd02 30T 27T 3.9T 88% /gl/lvosb03pool05vd02
/dev/mapper/vgosb03pool05vd03-lvosb03pool05vd03 30T 27T 3.9T 88% /gl/lvosb03pool05vd03
/dev/mapper/vgosb03pool05vd04-lvosb03pool05vd04 31T 27T 3.9T 88% /gl/lvosb03pool05vd04
--
Dr. Stefan Solbrig
Universität Regensburg, Fakultät für Physik,
93040 Regensburg, Germany
Tel +49-941-943-2097