Re: rbd map issues: no such file or directory (ENOENT) AND map wrong image

[David Zafman <david.zafman@xxxxxxxxxxx>]

Transferring this back the ceph-users.  Sorry, I can't help with rbd issues.  One thing I will say is that if you are mounting an rbd device with a filesystem on a machine to export ftp, you can't also export the same device via iSCSI.

David Zafman

Senior Developer

http://www.inktank.com

On Aug 19, 2013, at 8:39 PM, PJ <linalin1979@xxxxxxxxx> wrote:

2013/8/14 David Zafman <david.zafman@xxxxxxxxxxx>

On Aug 12, 2013, at 7:41 PM, Josh Durgin <josh.durgin@xxxxxxxxxxx> wrote:

> On 08/12/2013 07:18 PM, PJ wrote:
>>

>> If the target rbd device only map on one virtual machine, format it as
>> ext4 and mount to two places
>>   mount /dev/rbd0 /nfs --> for nfs server usage
>>   mount /dev/rbd0 /ftp  --> for ftp server usage
>> nfs and ftp servers run on the same virtual machine. Will file system
>> (ext4) help to handle the simultaneous access from nfs and ftp?
> 

> I doubt that'll work perfectly on a normal disk, although rbd should
> behave the same in this case. Consider what happens when to be some
> issues when the same files are modified at once by the ftp and nfs
> servers. You could run ftp on an nfs client on a different machine
> safely.
>

Modern Linux kernels will do a bind mount when a block device is mounted on 2 different directories.   Think directory hard links.  Simultaneous access will NOT corrupt ext4, but as Josh said modifying the same file at once by ftp and nfs isn't going produce good results.  With file locking 2 nfs clients could coordinate using advisory locking.

David Zafman
Senior Developer
http://www.inktank.com

The first issue is reproduced, but there are changes to system configuration. Due to hardware shortage, we only have one physical machine installed one OSD and runs 6 virtual machines on it. Only one monitor (wistor-003) and one FTP server (wistor-004), the other virtual machines are iSCSI servers.

The log size is big because when enable FTP service for a rbd device, we have a rbd map retry loop in case it fails (retry rbd map every 10 sec and last for 3 minutes). Please download monitor log from below link,

https://www.dropbox.com/s/88cb9q91cjszuug/ceph-mon.wistor-003.log.zip

Here are the operation steps:

1. The pool rex is created

       Around 2013-08-20 09:16:38~09:16:39

2. The first time to map rbd device on wistor-004 and it fails (all retries failed)

       Around 2013-08-20 09:17:43~09:20:46 (180 sec)

3. Tried second time and it works, but still have 9 fails in retry loop

       Around 2013-08-20 09:20:48~09:22:10 (82 sec)

_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com