dmesg output please. 2014-08-11 2:16 GMT+04:00 Lorieri <lorieri at gmail.com>: > same here, did you manage to fix it ? > > On Mon, Oct 28, 2013 at 3:13 PM, Kevin Weiler > <Kevin.Weiler at imc-chicago.com> wrote: > > Hi Josh, > > > > We did map it directly to the host, and it seems to work just fine. I > > think this is a problem with how the container is accessing the rbd > module. > > > > -- > > > > Kevin Weiler > > > > IT > > > > > > IMC Financial Markets | 233 S. Wacker Drive, Suite 4300 | Chicago, IL > > 60606 | http://imc-chicago.com/ > > > > Phone: +1 312-204-7439 | Fax: +1 312-244-3301 | E-Mail: > > kevin.weiler at imc-chicago.com > > > > > > > > > > > > > > > > On 10/18/13 7:50 PM, "Josh Durgin" <josh.durgin at inktank.com> wrote: > > > >>On 10/18/2013 10:04 AM, Kevin Weiler wrote: > >>> The kernel is 3.11.4-201.fc19.x86_64, and the image format is 1. I did, > >>> however, try a map with an RBD that was format 2. I got the same error. > >> > >>To rule out any capability drops as the culprit, can you map an rbd > >>image on the same host outside of a container? > >> > >>Josh > >> > >>> -- > >>> > >>> *Kevin Weiler* > >>> > >>> IT > >>> > >>> IMC Financial Markets | 233 S. Wacker Drive, Suite 4300 | Chicago, IL > >>> 60606 | http://imc-chicago.com/ > >>> > >>> Phone: +1 312-204-7439 | Fax: +1 312-244-3301 | E-Mail: > >>> _kevin.weiler at imc-chicago.com <mailto:Kevin.Weiler at imc-chicago.com>_ > >>> > >>> > >>> From: Gregory Farnum <greg at inktank.com <mailto:greg at inktank.com>> > >>> Date: Friday, October 18, 2013 10:26 AM > >>> To: Omar Marquez <Omar.Marquez at imc-chicago.com > >>> <mailto:Omar.Marquez at imc-chicago.com>> > >>> Cc: Kyle Bader <kyle.bader at gmail.com <mailto:kyle.bader at gmail.com>>, > >>> Kevin Weiler <kevin.weiler at imc-chicago.com > >>> <mailto:kevin.weiler at imc-chicago.com>>, "ceph-users at lists.ceph.com > >>> <mailto:ceph-users at lists.ceph.com>" <ceph-users at lists.ceph.com > >>> <mailto:ceph-users at lists.ceph.com>>, Khalid Goudeaux > >>> <Khalid.Goudeaux at imc-chicago.com > >>><mailto:Khalid.Goudeaux at imc-chicago.com>> > >>> Subject: Re: [ceph-users] mounting RBD in linux containers > >>> > >>> What kernel are you running, and which format is the RBD image? I > >>> thought we had a special return code for when the kernel doesn't > support > >>> the features used by that image, but that could be the problem. > >>> -Greg > >>> > >>> On Thursday, October 17, 2013, Omar Marquez wrote: > >>> > >>> > >>> Strace produces below: > >>> > >>> ? > >>> > >>> futex(0xb5637c, FUTEX_WAKE_OP_PRIVATE, 1, 1, 0xb56378, > >>> {FUTEX_OP_SET, 0, FUTEX_OP_CMP_GT, 1}) = 1 > >>> futex(0xb562f8, FUTEX_WAKE_PRIVATE, 1) = 1 > >>> add_key(0x424408, 0x7fff82c4e210, 0x7fff82c4e140, 0x22, > >>> 0xfffffffe) = 607085216 > >>> stat("/sys/bus/rbd", {st_mode=S_IFDIR|0755, st_size=0, ...}) = > 0 > >>> *open("/sys/bus/rbd/add", O_WRONLY) = 3* > >>> *write(3, "10.198.41.6:6789 > >>> <http://10.198.41.6:6789>,10.198.41.8:678 > >>> <http://10.198.41.8:678>"..., 96) = -1 EINVAL (Invalid > >>>argument)* > >>> close(3) = 0 > >>> rt_sigaction(SIGINT, {SIG_IGN, [], SA_RESTORER, > 0x7fbf8a7efa90}, > >>> {SIG_DFL, [], 0}, 8) = 0 > >>> rt_sigaction(SIGQUIT, {SIG_IGN, [], SA_RESTORER, > >>> 0x7fbf8a7efa90}, {SIG_DFL, [], 0}, 8) = 0 > >>> rt_sigprocmask(SIG_BLOCK, [CHLD], [PIPE], 8) = 0 > >>> clone(child_stack=0, flags=CLONE_PARENT_SETTID|SIGCHLD, > >>> parent_tidptr=0x7fff82c4e040) = 22 > >>> wait4(22, [{WIFEXITED(s) && WEXITSTATUS(s) == 0}], 0, NULL) = > 22 > >>> rt_sigaction(SIGINT, {SIG_DFL, [], SA_RESTORER, > 0x7fbf8a7efa90}, > >>> NULL, 8) = 0 > >>> rt_sigaction(SIGQUIT, {SIG_DFL, [], SA_RESTORER, > >>> 0x7fbf8a7efa90}, NULL, 8) = 0 > >>> rt_sigprocmask(SIG_SETMASK, [PIPE], NULL, 8) = 0 > >>> write(2, "rbd: add failed: ", 17rbd: add failed: ) = 17 > >>> write(2, "(22) Invalid argument", 21(22) Invalid argument) = > >>>21 > >>> write(2, "\n", 1 > >>> ) = 1 > >>> exit_group(1) = ? > >>> +++ exited with 1 +++ > >>> > >>> > >>> The app is run inside the container with setuid = 0 and the > >>> container is able to mount all required filesystems ? could this > >>> still be a capability problem ? Also I do not see any call to > >>> capset() in the strafe log ? > >>> > >>> -- > >>> Om > >>> > >>> > >>> From: Kyle Bader <kyle.bader at gmail.com> > >>> Date: Thursday, October 17, 2013 5:08 PM > >>> To: Kevin Weiler <Kevin.Weiler at imc-chicago.com> > >>> Cc: "ceph-users at lists.ceph.com" <ceph-users at lists.ceph.com>, Omar > >>> Marquez <omar.marquez at imc-chicago.com>, Khalid Goudeaux > >>> <Khalid.Goudeaux at imc-chicago.com> > >>> Subject: Re: [ceph-users] mounting RBD in linux containers > >>> > >>> My first guess would be that it's due to LXC dropping capabilities, > >>> I'd investigate whether CAP_SYS_ADMIN is being dropped. You need > >>> CAP_SYS_ADMIN for mount and block ioctls, if the container doesn't > >>> have those privs a map will likely fail. Maybe try tracing the > >>> command with strace? > >>> > >>> On Thu, Oct 17, 2013 at 2:45 PM, Kevin Weiler > >>> <Kevin.Weiler at imc-chicago.com> wrote: > >>> > >>> Hi all, > >>> > >>> We're trying to mount an rbd image inside of a linux container > >>> that has been created with docker (https://www.docker.io/). We > >>> seem to have access to the rbd kernel module from inside the > >>> container: > >>> > >>> # lsmod | grep ceph > >>> libceph 218854 1 rbd > >>> libcrc32c 12603 3 xfs,libceph,dm_persistent_data > >>> > >>> And we can query the pool for available rbds and create rbds > >>> from inside the container: > >>> > >>> # rbd -p dockers --id dockers --keyring > >>> /etc/ceph/ceph.client.dockers.keyring create lxctest --size > >>>51200 > >>> # rbd -p dockers --id dockers --keyring > >>> /etc/ceph/ceph.client.dockers.keyring ls > >>> lxctest > >>> > >>> But for some reason, we can't seem to map the device to the > >>> container: > >>> > >>> # rbd -p dockers --id dockers --keyring > >>> /etc/ceph/ceph.client.dockers.keyring map lxctest > >>> rbd: add failed: (22) Invalid argument > >>> > >>> I don't see anything particularly interesting in dmesg or > >>> messages on either the container or the host box. Any ideas on > >>> how to troubleshoot this? > >>> > >>> Thanks! > >>> > >>> > >>> -- > >>> > >>> *Kevin Weiler* > >>> > >>> IT > >>> > >>> IMC Financial Markets | 233 S. Wacker Drive, Suite 4300 | > >>> Chicago, IL 60606 | http://imc-chicago.com/ > >>> > >>> Phone: +1 312-204-7439 | Fax: +1 312-244-3301 | E-Mail: > >>> _kevin.weiler at imc-chicago.com_ > >>> > >>> > >>> > >>>------------------------------------------------------------------------ > >>> > >>> The information in this e-mail is intended only for the person > >>> or entity to which it is addressed. > >>> > >>> It may contain confidential and /or privileged material. If > >>> someone other than the intended recipient should receive this > >>> e-mail, he / she shall not be entitled to read, disseminate, > >>> disclose or duplicate it. > >>> > >>> If you receive this e-mail unintentionally, please inform us > >>> immediately by "re > >>> > >>> > >>> > >>>------------------------------------------------------------------------ > >>> > >>> The information in this e-mail is intended only for the person or > >>> entity to which it is addressed. > >>> > >>> It may contain confidential and /or privileged material. If someone > >>> other than the intended recipient should receive this e-mail, he / > >>> she shall not be entitled to read, disseminate, disclose or > >>> duplicate it. > >>> > >>> If you receive this e-mail unintentionally, please inform us > >>> immediately by "reply" and then delete it from your system. > Although > >>> this information has been compiled with great care, neither IMC > >>> Financial Markets & Asset Management nor any of its related > entities > >>> shall accept any responsibility for any errors, omissions or other > >>> inaccuracies in this information or for the consequences thereof, > >>> nor shall it be bound in any way by the contents of this e-mail or > >>> its attachments. In the event of incomplete or incorrect > >>> transmission, please return the e-mail to the sender and > permanently > >>> delete this message and any attachments. > >>> > >>> Messages and attachments are scanned for all known viruses. Always > >>> scan attachments before opening them. > >>> > >>> > >>> > >>> -- > >>> Software Engineer #42 @ http://inktank.com | http://ceph.com > >>> > >>> > ------------------------------------------------------------------------ > >>> > >>> The information in this e-mail is intended only for the person or > entity > >>> to which it is addressed. > >>> > >>> It may contain confidential and /or privileged material. If someone > >>> other than the intended recipient should receive this e-mail, he / she > >>> shall not be entitled to read, disseminate, disclose or duplicate it. > >>> > >>> If you receive this e-mail unintentionally, please inform us > immediately > >>> by "reply" and then delete it from your system. Although this > >>> information has been compiled with great care, neither IMC Financial > >>> Markets & Asset Management nor any of its related entities shall accept > >>> any responsibility for any errors, omissions or other inaccuracies in > >>> this information or for the consequences thereof, nor shall it be bound > >>> in any way by the contents of this e-mail or its attachments. In the > >>> event of incomplete or incorrect transmission, please return the e-mail > >>> to the sender and permanently delete this message and any attachments. > >>> > >>> Messages and attachments are scanned for all known viruses. Always scan > >>> attachments before opening them. > >>> > >>> > >>> _______________________________________________ > >>> ceph-users mailing list > >>> ceph-users at lists.ceph.com > >>> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > >>> > >> > > > > > > ________________________________ > > > > The information in this e-mail is intended only for the person or entity > to which it is addressed. > > > > It may contain confidential and /or privileged material. If someone > other than the intended recipient should receive this e-mail, he / she > shall not be entitled to read, disseminate, disclose or duplicate it. > > > > If you receive this e-mail unintentionally, please inform us immediately > by "reply" and then delete it from your system. Although this information > has been compiled with great care, neither IMC Financial Markets & Asset > Management nor any of its related entities shall accept any responsibility > for any errors, omissions or other inaccuracies in this information or for > the consequences thereof, nor shall it be bound in any way by the contents > of this e-mail or its attachments. In the event of incomplete or incorrect > transmission, please return the e-mail to the sender and permanently delete > this message and any attachments. > > > > Messages and attachments are scanned for all known viruses. Always scan > attachments before opening them. > > _______________________________________________ > > ceph-users mailing list > > ceph-users at lists.ceph.com > > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > _______________________________________________ > ceph-users mailing list > ceph-users at lists.ceph.com > http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com > -- ? ?????????, ??????? ???? ??????????? ???.: +79229045757 -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.ceph.com/pipermail/ceph-users-ceph.com/attachments/20140811/2e5c6a40/attachment.htm>
 |