This is just a complicated version of the following test:
with one file
for n in N:
pick one of two clients
on that client:
open the file, increment a value, close the file
print the value after it was incremented.
check that the incremented value is equal to n+1
It seems like when run, this test sees the counter is going back to
zero, which I would guess is PHP's behaviour when for some reason it
fails to open/load its file.
PHP is relying on flock() for safety here[1]. A quick google for
"cephfs flock" takes me to http://tracker.ceph.com/issues/2825.
Almost certainly worth re-testing with latest Ceph.
John
1. https://github.com/php/php-src/blob/d9bfe06194ae8f760cb43a3e7120d0503f327398/ext/session/mod_files.c
On Thu, Nov 14, 2013 at 4:11 PM, Gregory Farnum <greg@xxxxxxxxxxx> wrote:
> [ Adding list back. ]
>
> I don't know php; I'm not sure what it means for "count" to drop to
> zero in your script. What erroneous behavior do you believe the
> filesystem is displaying?
> -Greg
> Software Engineer #42 @ http://inktank.com | http://ceph.com
>
> On Thu, Nov 14, 2013 at 12:04 AM, M. Piscaer <debian@xxxxxxxxxxx> wrote:
>> Greg,
>>
>> What do you mean,
>>
>> What info do you need?
>>
>> Kind regards,
>>
>> Michiel Piscaer
>>
>>
>> On wo, 2013-11-13 at 16:19 -0800, Gregory Farnum wrote:
>>> I'm not too familiar with the toolchain you're using, so can you
>>> clarify what problem you're seeing with CephFS here?
>>> -Greg
>>> Software Engineer #42 @ http://inktank.com | http://ceph.com
>>>
>>>
>>> On Wed, Nov 13, 2013 at 12:06 PM, M. Piscaer <debian@xxxxxxxxxxx> wrote:
>>> >
>>> > Hi,
>>> >
>>> > I have an webcluster setup, where on the loadbalancers the persistence
>>> > timeout is 0. To share the sessions I use ceph version 0.56.7, like you
>>> > see on the diagram.
>>> >
>>> > +----------------+
>>> > | Internet |
>>> > +----------------+
>>> > |
>>> > +-----+-----------------------+
>>> > | |
>>> > +-----------------+ +-----------------+
>>> > | loadbalancer-01 | | loadbalancer-02 |
>>> > +-----------------+ +-----------------+
>>> > | |
>>> > +-----+--192.168.1.0/24-------+
>>> > |
>>> > +--------+ | +--------+
>>> > | web-01 |---+-----| web-02 |
>>> > +--------+ | +--------+
>>> > |
>>> > +--------+ | +--------+
>>> > | osd-01 |---+-----| osd-02 |
>>> > +--------+ | +--------+
>>> > |
>>> > +--------+ | +--------+
>>> > | mds-01 |---+-----| mds-02 |
>>> > +--------+ | +--------+
>>> > |
>>> > +--------+--------+-------------------+
>>> > | | |
>>> > +--------+ +--------+ +--------+
>>> > | mon-01 | | mon-02 | | mon-03 |
>>> > +--------+ +--------+ +--------+
>>> >
>>> > I mount on the web nodes the ceph mds tables:
>>> > ** /etc/fstab **
>>> > mon-01:/ < Session_mountpoint> ceph
>>> > defaults,name=admin,secret=<secret_key> 0 0
>>> >
>>> > My probem is that when the sessions gets a frequent update, I sometimes
>>> > loss my session data.
>>> >
>>> > I can reproduse my problem with the following PHP script:
>>> >
>>> > <?php
>>> > // page2.php
>>> >
>>> > session_save_path('/var/www/storage/sessions/');
>>> > session_start();
>>> >
>>> > $_SESSION['count']++;
>>> > echo 'count: ';
>>> > echo $_SESSION['count'];
>>> >
>>> > ?>
>>> >
>>> > When i run the following commands:
>>> >
>>> > michielp@michielp-hp:~$ wget --no-check-certificate
>>> > --keep-session-cookies --save-cookies /tmp/cookies.txt
>>> > https://sub.domain.nl/page2.php -O -
>>> > michielp@michielp-hp:~$ for foo in {1..10000}; do wget
>>> > --no-check-certificate --load-cookies /tmp/cookies.txt
>>> > "https://sub.domain.nl/page2.php"; -O - -o /dev/null; sleep 0.3; done
>>> >
>>> > At 10, 100 and 1000 and further the couter hits to 0. When i use sleep
>>> > 0.4 everthing works fine.
>>> >
>>> > michielp@michielp-hp:~$ for foo in {1..10000}; do wget
>>> > --no-check-certificate --load-cookies /tmp/cookies.txt
>>> > "https://sub.domain.nl/page2.php"; -O - -o /dev/null; done
>>> >
>>> > count: 1
>>> > count: 2
>>> > count: 3
>>> > count: 4
>>> > count: 5
>>> > count: 6
>>> > count: 7
>>> > count: 8
>>> > count: 9
>>> > count: 10
>>> > count: 1
>>> > count: 2
>>> > count: 1
>>> >
>>> > Also when is switch off one of the webservers the problem disappears.
>>> >
>>> > On mds-01 is see the folllowing message:
>>> > root@isp-oscaccstormds-01:/var/log/ceph# tail ceph-mds.5.log
>>> > 2013-11-13 20:49:00.428592 7f20fca22700 0 mds.0.server
>>> > handle_client_file_setlock: start: 0, length: 0, client: 18900, pid:
>>> > 10032, type: 4
>>> >
>>> > The config of the ceph cluster looks like:
>>> >
>>> > [global]
>>> > auth cluster required = cephx
>>> > auth service required = cephx
>>> > auth client required = cephx
>>> > keyring = /etc/ceph/keyring
>>> > cluster network = 192.168.1.0/24
>>> > public network = 192.168.1.0/24
>>> >
>>> > fsid = 82ecbd50-81ff-4f6c-a009-0bd02a1b4043
>>> >
>>> > [mon]
>>> > mon data = /var/lib/ceph/mon/mon.$id
>>> >
>>> > [osd]
>>> > osd journal size = 4096
>>> > filestore flusher = false
>>> > osd data = /var/lib/ceph/osd/osd.$id
>>> > osd journal = /var/lib/ceph/osd/osd.$id/journal
>>> > osd mkfs type = xfs
>>> > keyring = /var/lib/ceph/osd/osd.$id/keyring
>>> >
>>> > [mds]
>>> > mds data = /var/lib/ceph/mds/mds.$id
>>> > keyring = /var/lib/ceph/mds/mds.$id/keyring
>>> >
>>> > [mon.0]
>>> > host = mon-01
>>> > mon addr = 192.168.1.56:6789
>>> >
>>> > [mon.1]
>>> > host = mon-02
>>> > mon addr = 192.168.1.57:6789
>>> >
>>> > [mon.2]
>>> > host = mon-03
>>> > mon addr = 192.168.1.58:6789
>>> >
>>> > [mds.5]
>>> > host = mds-01
>>> >
>>> > [mds.6]
>>> > host = mds-02
>>> >
>>> > [osd.0]
>>> > host = osd-02
>>> > devs = /dev/sdb1
>>> > cluster addr = 192.168.1.60
>>> > public addr = 192.168.1.60
>>> >
>>> > [osd.1]
>>> > host = osd-01
>>> > devs = /dev/sdb1
>>> > cluster addr = 192.168.1.59
>>> > public addr = 192.168.1.59
>>> >
>>> >
>>> > Kinds regards,
>>> >
>>> > Michiel Piscaer
>>> >
>>> >
>>> > _______________________________________________
>>> > ceph-users mailing list
>>> > ceph-users@xxxxxxxxxxxxxx
>>> > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>>
>>
> _______________________________________________
> ceph-users mailing list
> ceph-users@xxxxxxxxxxxxxx
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
