Re: locking on mds

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



I'm not too familiar with the toolchain you're using, so can you
clarify what problem you're seeing with CephFS here?
-Greg
Software Engineer #42 @ http://inktank.com | http://ceph.com


On Wed, Nov 13, 2013 at 12:06 PM, M. Piscaer <debian@xxxxxxxxxxx> wrote:
>
> Hi,
>
> I have an webcluster setup, where on the loadbalancers the persistence
> timeout is 0. To share the sessions I use ceph version 0.56.7, like you
> see on the diagram.
>
> +----------------+
> | Internet       |
> +----------------+
>              |
>        +-----+-----------------------+
>        |                             |
> +-----------------+          +-----------------+
> | loadbalancer-01 |          | loadbalancer-02 |
> +-----------------+          +-----------------+
>        |                             |
>        +-----+--192.168.1.0/24-------+
>              |
> +--------+   |     +--------+
> | web-01 |---+-----| web-02 |
> +--------+   |      +--------+
>              |
> +--------+   |     +--------+
> | osd-01 |---+-----| osd-02 |
> +--------+   |     +--------+
>              |
> +--------+   |     +--------+
> | mds-01 |---+-----| mds-02 |
> +--------+   |     +--------+
>              |
>     +--------+--------+-------------------+
>     |                 |                   |
> +--------+         +--------+         +--------+
> | mon-01 |         | mon-02 |         | mon-03 |
> +--------+         +--------+         +--------+
>
> I mount on the web nodes the ceph mds tables:
> ** /etc/fstab **
> mon-01:/        < Session_mountpoint>   ceph
> defaults,name=admin,secret=<secret_key> 0       0
>
> My probem is that when the sessions gets a frequent update, I sometimes
> loss my session data.
>
> I can reproduse my problem with the following PHP script:
>
> <?php
> // page2.php
>
> session_save_path('/var/www/storage/sessions/');
> session_start();
>
> $_SESSION['count']++;
> echo 'count: ';
> echo $_SESSION['count'];
>
> ?>
>
> When i run the following commands:
>
> michielp@michielp-hp:~$ wget --no-check-certificate
> --keep-session-cookies --save-cookies /tmp/cookies.txt
> https://sub.domain.nl/page2.php -O -
> michielp@michielp-hp:~$ for foo in {1..10000}; do wget
> --no-check-certificate --load-cookies /tmp/cookies.txt
> "https://sub.domain.nl/page2.php"; -O - -o /dev/null; sleep 0.3; done
>
> At 10, 100 and 1000 and further the couter hits to 0. When i use sleep
> 0.4 everthing works fine.
>
> michielp@michielp-hp:~$ for foo in {1..10000}; do wget
> --no-check-certificate --load-cookies /tmp/cookies.txt
> "https://sub.domain.nl/page2.php"; -O - -o /dev/null; done
>
> count: 1
> count: 2
> count: 3
> count: 4
> count: 5
> count: 6
> count: 7
> count: 8
> count: 9
> count: 10
> count: 1
> count: 2
> count: 1
>
> Also when is switch off one of the webservers the problem disappears.
>
> On mds-01 is see the folllowing message:
> root@isp-oscaccstormds-01:/var/log/ceph# tail ceph-mds.5.log
> 2013-11-13 20:49:00.428592 7f20fca22700  0 mds.0.server
> handle_client_file_setlock: start: 0, length: 0, client: 18900, pid:
> 10032, type: 4
>
> The config of the ceph cluster looks like:
>
> [global]
>   auth cluster required = cephx
>   auth service required = cephx
>   auth client required = cephx
>   keyring = /etc/ceph/keyring
>   cluster network = 192.168.1.0/24
>   public network = 192.168.1.0/24
>
>   fsid = 82ecbd50-81ff-4f6c-a009-0bd02a1b4043
>
> [mon]
>   mon data = /var/lib/ceph/mon/mon.$id
>
> [osd]
>   osd journal size = 4096
>   filestore flusher = false
>   osd data = /var/lib/ceph/osd/osd.$id
>   osd journal = /var/lib/ceph/osd/osd.$id/journal
>   osd mkfs type = xfs
>   keyring = /var/lib/ceph/osd/osd.$id/keyring
>
> [mds]
>   mds data = /var/lib/ceph/mds/mds.$id
>   keyring = /var/lib/ceph/mds/mds.$id/keyring
>
> [mon.0]
>   host = mon-01
>   mon addr = 192.168.1.56:6789
>
> [mon.1]
>   host = mon-02
>   mon addr = 192.168.1.57:6789
>
> [mon.2]
>   host = mon-03
>   mon addr = 192.168.1.58:6789
>
> [mds.5]
>   host = mds-01
>
> [mds.6]
>   host = mds-02
>
> [osd.0]
>    host = osd-02
>    devs = /dev/sdb1
>    cluster addr = 192.168.1.60
>    public addr = 192.168.1.60
>
> [osd.1]
>    host = osd-01
>    devs = /dev/sdb1
>    cluster addr = 192.168.1.59
>    public addr = 192.168.1.59
>
>
> Kinds regards,
>
> Michiel Piscaer
>
>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com




[Index of Archives]     [Information on CEPH]     [Linux Filesystem Development]     [Ceph Development]     [Ceph Large]     [Ceph Dev]     [Linux USB Development]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]     [xfs]


  Powered by Linux