locking on mds

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Hi,

I have an webcluster setup, where on the loadbalancers the persistence
timeout is 0. To share the sessions I use ceph version 0.56.7, like you
see on the diagram.

+----------------+ 
| Internet       |
+----------------+
             |
       +-----+-----------------------+
       |                             |
+-----------------+          +-----------------+ 
| loadbalancer-01 |          | loadbalancer-02 |
+-----------------+          +-----------------+
       |                             |
       +-----+--192.168.1.0/24-------+
             |
+--------+   |     +--------+
| web-01 |---+-----| web-02 |
+--------+   |      +--------+
             |
+--------+   |     +--------+
| osd-01 |---+-----| osd-02 |
+--------+   |     +--------+
             |
+--------+   |     +--------+
| mds-01 |---+-----| mds-02 |
+--------+   |     +--------+
             |
    +--------+--------+-------------------+
    |                 |                   |
+--------+         +--------+         +--------+
| mon-01 |         | mon-02 |         | mon-03 |
+--------+         +--------+         +--------+

I mount on the web nodes the ceph mds tables:
** /etc/fstab **
mon-01:/	< Session_mountpoint>	ceph
defaults,name=admin,secret=<secret_key> 0	0

My probem is that when the sessions gets a frequent update, I sometimes
loss my session data.

I can reproduse my problem with the following PHP script:

<?php
// page2.php

session_save_path('/var/www/storage/sessions/');
session_start();

$_SESSION['count']++;
echo 'count: ';
echo $_SESSION['count'];

?>

When i run the following commands:

michielp@michielp-hp:~$ wget --no-check-certificate
--keep-session-cookies --save-cookies /tmp/cookies.txt
https://sub.domain.nl/page2.php -O -
michielp@michielp-hp:~$ for foo in {1..10000}; do wget
--no-check-certificate --load-cookies /tmp/cookies.txt
"https://sub.domain.nl/page2.php"; -O - -o /dev/null; sleep 0.3; done

At 10, 100 and 1000 and further the couter hits to 0. When i use sleep
0.4 everthing works fine.

michielp@michielp-hp:~$ for foo in {1..10000}; do wget
--no-check-certificate --load-cookies /tmp/cookies.txt
"https://sub.domain.nl/page2.php"; -O - -o /dev/null; done

count: 1
count: 2
count: 3
count: 4
count: 5
count: 6
count: 7
count: 8
count: 9
count: 10
count: 1
count: 2
count: 1

Also when is switch off one of the webservers the problem disappears.

On mds-01 is see the folllowing message:
root@isp-oscaccstormds-01:/var/log/ceph# tail ceph-mds.5.log
2013-11-13 20:49:00.428592 7f20fca22700  0 mds.0.server
handle_client_file_setlock: start: 0, length: 0, client: 18900, pid:
10032, type: 4

The config of the ceph cluster looks like:

[global]
  auth cluster required = cephx
  auth service required = cephx
  auth client required = cephx
  keyring = /etc/ceph/keyring
  cluster network = 192.168.1.0/24
  public network = 192.168.1.0/24

  fsid = 82ecbd50-81ff-4f6c-a009-0bd02a1b4043

[mon]
  mon data = /var/lib/ceph/mon/mon.$id

[osd]
  osd journal size = 4096
  filestore flusher = false
  osd data = /var/lib/ceph/osd/osd.$id
  osd journal = /var/lib/ceph/osd/osd.$id/journal
  osd mkfs type = xfs
  keyring = /var/lib/ceph/osd/osd.$id/keyring

[mds]
  mds data = /var/lib/ceph/mds/mds.$id
  keyring = /var/lib/ceph/mds/mds.$id/keyring

[mon.0]
  host = mon-01
  mon addr = 192.168.1.56:6789

[mon.1]
  host = mon-02
  mon addr = 192.168.1.57:6789

[mon.2]
  host = mon-03
  mon addr = 192.168.1.58:6789

[mds.5]
  host = mds-01

[mds.6]
  host = mds-02

[osd.0]
   host = osd-02
   devs = /dev/sdb1
   cluster addr = 192.168.1.60
   public addr = 192.168.1.60

[osd.1]
   host = osd-01
   devs = /dev/sdb1
   cluster addr = 192.168.1.59
   public addr = 192.168.1.59


Kinds regards,

Michiel Piscaer

Attachment: signature.asc
Description: This is a digitally signed message part

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux