Re: Fedora 15: a NFSv4 to glusterfs migration HOW-TO

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Excellent how to Dr.
Can you add this to the Fedora wiki?

2011/8/25 Dr. Michael J. Chudobiak <mjc@xxxxxxxxxxxxxxx>
As detailed in another thread, we upgraded a few test machines on our
LAN to Fedora 15 (with gnome-shell and firefox), with user folders
served from a NFSv4 server (F14 originally, then F15).

It just didn't work. The F15 desktops would freeze frequently. And
worse, this would freeze ALL desktops on the LAN intermittently, as the
NFS server struggled with client flakiness.

When it did work, Firefox would lose authenticated logins randomly,
presumably due to corruption of its cookies.sqlite file. sqlite and NFS
seems to be a nightmare, for both NFSv3 and NFSv4.

Moving from a NFSv4 server to a glusterfs server solved all of these
problems, and sped up boot times significantly too. glusterfs looks
intimidating at first, because of all its fancy replicating features and
what-not, but it turns out to be trivially easy to set up a simple
server than will replace 95% of the NFS installations out there.

Luckily, you can easily point both the NFS daemon and the glusterfs
daemon at the same export folder, so you can migrate clients slowly over
time.

This HOW-TO is intended to document the process. There are other similar
HOW-TOs out there, but they are all a little out-of-date or don't show
how to enable locking correctly, which is critical for Firefox.

In this example, we export the server's /fileserver folder, and mount it
on /fileserver on the clients. In my server, /fileserver was already
being served by the NFSv4 server, which is fine.

--------------------------------------------
1. On the server:
- yum install glusterfs-server

--------------------------------------------
2. On each client:
- yum install glusterfs-fuse
- mkdir /etc/glusterfs/
- mkdir /fileserver

--------------------------------------------
3. On the server, edit the volume configuration file
(/etc/glusterfs/glusterfsd.vol) so that it looks like this:

volume raw
  type storage/posix
  option directory /fileserver
end-volume

volume brick
  type features/posix-locks
  subvolumes raw
end-volume

volume server
  type protocol/server
  option transport-type tcp
  subvolumes brick
  option auth.addr.brick.allow *
end-volume


The first stanza selects the basic folder to export.

The second stanza adds file locking to it. This is required to support
Firefox, and some other applications.

The third stanza authorizes everyone to access this file-locked export
over the network.

There is also a /etc/glusterfs/glusterfsd.vol file on the system, for
configuring the management interface. For this simple installation it
does not need to be modified.

--------------------------------------------
4. Restart the server services:
- service glusterd restart
- service glusterfsd restart

I believe the first service is a management service, and the second is
the actual file-export service.
--------------------------------------------

--------------------------------------------
5. On the client, create the /etc/glusterfs/glusterfs.vol configuration
file, which should look like this:

volume client
  type protocol/client
  option transport-type tcp
  option remote-host 192.168.0.3  # use YOUR server IP here
  option remote-subvolume brick
end-volume


--------------------------------------------
6. On the client, add this line to the end of /etc/rc.d/rc.local:

mount -t glusterfs /etc/glusterfs/glusterfs.vol /fileserver

--------------------------------------------
7. On the client, reboot, and check /var/log/messages for errors. On one
machine, we had an selinux problem that was flagged in the logs. We had
to manually create the logging file using:

touch /var/log/glusterfs/fileserver.log; reboot

--------------------------------------------
8. On the client, see if you can access the files in /fileserver. If
not, read the /var/log/glusterfs/* files on both the client and the server.

At this point, everything should work!



Weird things and gotchas:

A. You need the file-locking option to make Firefox work properly.

B. LibreOffice wouldn't start on one system, until we did:
rm ~/.libreoffice
rm ~/.openoffice.org

C. selinux prevented the creation of log files on one client, which
prevented the filesystem from mouting. The manual fix noted above fixed
that.

D. This HOWTO mounts the glusterfs from /etc/rc.d/rc.local, which is the
last step in the boot process. In theory, you can mount it from
/etc/fstab or using autofs. However, we found that autofs mounting just
didn't work - not sure why. fstab mounting didn't work either - I
suspect it occurred too early in the boot process. /etc needs to be up
and running so glusterfs can read the config file, and I don't think the
current init/systemd files handle this correctly. There are some Debian
bug reports about this that you can google.


I hope this is useful to someone, and that we can finally drive a stake
through the heart of NFS...


- Mike
--
users mailing list
users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines



--
--
Marcos Luis Ortíz Valmaseda
 Software Engineer (UCI)
 Linux User # 418229
 http://marcosluis2186.posterous.com
 http://www.linkedin.com/in/marcosluis2186
 https://fedoraproject.org/wiki/User:Marcosluis

-- 
users mailing list
users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe or change subscription options:
https://admin.fedoraproject.org/mailman/listinfo/users
Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines
[Index of Archives]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [EPEL Devel]     [Fedora Magazine]     [Fedora Summer Coding]     [Fedora Laptop]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Desktop]     [Fedora Fonts]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Yosemite News]     [Gnome Users]     [KDE Users]     [Fedora Art]     [Fedora Docs]     [Fedora Sparc]     [Libvirt Users]     [Fedora ARM]

  Powered by Linux