Re: NFS stale-filehandles

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



>>T. Horsnell wrote:
>>> FC6, NFS V3
>>> -----------
>>> 
>>> I'm trying to get to the state where
>>>  
>>> 1. I can have a bunch of NFS clients which have mounted
>>>    filesystems exported by a fileserver,
>>> 
>>> 2. The fileserver is stopped, the (SCSI) disks containing
>>>    the filesystems are moved to different positions on the SCSI bus,
>>>    and the fileserver is rebooted.
>>> 
>>> 3. The clients can still continue to access the correct filesystems,
>>>    without having to umount/re-mount anything, even though their position
>>>    on the server's SCSI bus has changed. 
>>> 
>>> The clients are members of a compute farm. They mount the NFS 
>>> filesystems with the 'hard' option so that their NFS requests 
>>> stall if the server is off for any reason, and so far this works
>>> well. However, if the server disks are moved, there are problems.
>>> 
>>> 
>>> I started off mounting the server filesystems by label, with lines
>>> in /etc/fstab  like:
>>> 
>>> LABEL=/filesys1		/filesystem1	ext3	defaults	0 2
>>> 
>>> This gave me pseudo-persistence in that wherever the disks were on
>>> the SCSI bus, they were always mounted on the correct mountpoint.
>>> With this setup, the underlying device (/dev/sda /dev/sdb etc)
>>> as shown by 'mount -t ext3' changed when the disk position changed.
>>> I then discovered that if I swapped two disks over on the server
>>> while those filesystems were still NFS-mounted by a client, the
>>> client didnt notice the swap, but continued to access the disks
>>> in the unswapped position, and hence access the wrong filesystem.
>>> There were no NFS complaints, just complaints from users.
>>> 
>>> 
>>> I'm now using a bunch of udev rules to give me device-name persistence
>>> instead of relying on the partition label, and I have lines in fstab like:
>>> 
>>> /dev/dsk0_1		/filesystem1	ext3	defaults	0 2
>>> 
>>> Now, wherever I shift a disk to on the server SCSI bus, the underlying
>>> device-name stays constant, but the client objects with a 'stale NFS filehandle'
>>> error when the disk-position is changed, and I have to umount and remount at
>>> the client. Its a slight improvement in that user-processes on the client
>>> cant inadvertently use the wrong filesystem, but I would much prefer it to
>>> be transparent. Is this possible with NFS
>>> 
>>> Cheers,
>>> Terry
>>> 
>>
>>Yeap.
>>
>>A client knows a filesystem by server ip address, and by fsid (see
>>man exports), the default generated fsid is based on the underlying
>>device id, so if the device id changes (moved on scsi bus)
>>then the fsid changes, and from the clients point of view the
>>filesystem is no longer there.   And you can get odd failures,
>>if someones "new" fsid matches someone elses "old" fsid you
>>will get interesting bad results, ie /home will say it is /home
>>on a client by the directories will look like /opt.   If you have
>>large fsids it is harder to get, but I have seen it happen.
>>
>>You can in the exports file set the fsid to something, and note
>>that you can get into this same issue using LVM, and just adjusting
>>the order of turning on the VG's (first gets id 1, second 2, changing
>>order changes this).    The fsids need to be unique across a
>>given server.
>>
>
>Aha! Many thanks for this pointer. I never noticed that option in the
>exports man page. I'll do some experiments straight away.

This is just the job. I dont even have to use udev to get
persistent device names if I use LABEL= in fstab. One question
though:

In the absence of an fsid= option, how does Linux compute a
filesystem handle? Unless I know this, or unless I provide
an fsid= option for *all* my exported filesystems, I risk
choosing an id which may collide with the Linux-generated one.

Thanks,
Terry

-- 
fedora-list mailing list
fedora-list@xxxxxxxxxx
To unsubscribe: https://www.redhat.com/mailman/listinfo/fedora-list
[Index of Archives]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [Fedora Magazine]     [Fedora News]     [Fedora Summer Coding]     [Fedora Laptop]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Maintainers]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Legacy]     [Fedora Desktop]     [Fedora Fonts]     [ATA RAID]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [SSH]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Centos]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Tux]     [Yosemite News]     [Gnome Users]     [KDE Users]     [Fedora Art]     [Fedora Docs]     [Asterisk PBX]     [Fedora Sparc]     [Fedora Universal Network Connector]     [Libvirt Users]     [Fedora ARM]

  Powered by Linux