Re: Passing LUA script via python rados execute

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Sorry about the confusion. Let us know if you have any issues / questions :)

On Thu, Feb 16, 2017 at 1:05 PM, Nick Fisk <nick@xxxxxxxxxx> wrote:
> Bingo!!
>
> That was it, I was just using the example from here
>
> http://noahdesu.github.io/2015/12/06/load-lua-rados-plugin-from-fs.html
>
> cmd = {
>   "script": """
>       function upper(input, output)
>         input_str = input:str()
>         upper_str = string.upper(input_str)
>         output:append(upper_str)
>       end
>       cls.register(upper)
>   """,
>   "handler": "upper",
>   "input": "this string was in lower case",
> }
>
> Thanks for all your help on this.
>
> Nick
>
>> -----Original Message-----
>> From: Noah Watkins [mailto:noahwatkins@xxxxxxxxx]
>> Sent: 16 February 2017 21:01
>> To: Nick Fisk <nick@xxxxxxxxxx>
>> Cc: ceph-devel <ceph-devel@xxxxxxxxxxxxxxx>
>> Subject: Re: Passing LUA script via python rados execute
>>
>> It looks like you might be using a script / example that was written before the
>> code was merged into Ceph and might be a little out of date. Use
>> `objclass.register` instead of `cls.register`. There is a test script at the top of
>> https://github.com/ceph/ceph/blob/master/src/test/cls_lua/test_cls_lua.cc
>> that you can use as a reference. If the test programs are installed you should
>> also have the executable `ceph_test_cls_lua` which will run the cls_lua unit
>> tests.
>>
>> On Thu, Feb 16, 2017 at 12:52 PM, Nick Fisk <nick@xxxxxxxxxx> wrote:
>> > Hi Noah,
>> >
>> > Thanks for the response.
>> >
>> > You were right about the need to enable the class in the ceph.conf. I did
>> see that note in the Kraken release notes, but assumed that as the LUA class
>> is in the Ceph source tree now, that it would have been included with all the
>> others in that directory. Anyway slap on the wrist for me not checking that.
>> >
>> > However, although that got me a bit further, I'm now getting a read
>> > error
>> >
>> > rados.IOError: Ioctx.read(rbd): failed to read test
>> >
>> > Which is a bit puzzling. As there is definitely an object called test on the rbd
>> pool and if I issue a ioctx.read right before the execute call, it returns
>> successfully with the objects contents. So I'm guessing this is actually the lua
>> class/script bombing out and not a read error?
>> >
>> > I also get this error in the OSD log
>> >
>> > 2017-02-16 20:41:45.065617 7f08da336700  0 <cls>
>> > /tmp/buildd/ceph-11.2.0/src/cls/lua/cls_lua.cc:1004: error: [string
>> > "..."]:7: attempt to index a nil value (global 'cls')
>> >
>> > Don't know if that rings any bells?
>> >
>> > Nick
>> >
>> >> -----Original Message-----
>> >> From: Noah Watkins [mailto:noahwatkins@xxxxxxxxx]
>> >> Sent: 15 February 2017 23:43
>> >> To: nick@xxxxxxxxxx
>> >> Cc: ceph-devel <ceph-devel@xxxxxxxxxxxxxxx>
>> >> Subject: Re: Passing LUA script via python rados execute
>> >>
>> >> Hi Nick,
>> >>
>> >> First thing to note is that in Kraken that object classes not
>> >> whitelisted need to be enabled explicitly. This is in the Kraken
>> >> release notes
>> >> (http://docs.ceph.com/docs/master/release-notes/):
>> >>
>> >> tldr: add 'osd class load list = *' and 'osd class default list = *'
>> >> to ceph.conf.
>> >>
>> >> - The ‘osd class load list’ config option is a list of object class
>> >> names that the OSD is permitted to load (or ‘*’ for all classes). By
>> >> default it contains all existing in-tree classes for backwards compatibility.
>> >>
>> >> - The ‘osd class default list’ config option is a list of object class names (or
>> ‘*’
>> >> for all classes) that clients may invoke having only the ‘*’, ‘x’,
>> >> ‘class-read’, or ‘class-write’ capabilities. By default it contains
>> >> all existing in-tree classes for backwards compatibility.
>> >> Invoking classes not listed in ‘osd class default list’ requires a
>> >> capability naming the class (e.g. ‘allow class foo’).
>> >>
>> >> I suspect that will resolve the issue.
>> >>
>> >> If you've done that and it still doesn't work then the next thing I'd
>> >> suggest is creating the target object before running the command.
>> >> Operations on objects that don't exist sometimes seem non-intuitive to
>> me.
>> >>
>> >> Let me know if that doesn't work and we can look at debugging further.
>> >>
>> >> Thanks,
>> >> - Noah
>> >>
>> >> On Wed, Feb 15, 2017 at 1:11 PM, Nick Fisk <nick@xxxxxxxxxx> wrote:
>> >> > Hi Noah,
>> >> >
>> >> > I'm trying to follow your example where you can pass a LUA script
>> >> > as json when calling the rados execute function in Python. However
>> >> > I'm getting a rados permission denied error saying its failed to
>> >> > read the test object I have placed on the pool.
>> >> >
>> >> > I have also tried calling the cls_hello object class and this
>> >> > works, so I believe its something to do with the LUA functionality
>> >> > that's causing it to bomb out. This is running on Kraken.
>> >> >
>> >> > Any ideas would be gratefully received.
>> >> >
>> >> > Nick
>> >> >
>> >> > Ie
>> >> > print json.dumps(cmd)
>> >> > ret, data = ioctx.execute('test', 'lua', 'eval_json',
>> >> > json.dumps(cmd))
>> >> >
>> >> > Outputs
>> >> > python radoslua.py --object-name=test --pool=rbd
>> >> > {"handler": "upper", "script": "\n      function upper(input, output)\n
>> >> > input_str = input:str()\n        upper_str = string.upper(input_str)\n
>> >> > output:append(upper_str)\n      end\n      cls.register(upper)\n  "}
>> >> > Traceback (most recent call last):
>> >> >   File "radoslua.py", line 47, in <module>
>> >> >     ret, data = ioctx.execute('test', 'lua', 'eval_json', json.dumps(cmd))
>> >> >   File "rados.pyx", line 451, in
>> >> > rados.requires.wrapper.validate_func
>> >> > (/tmp/buildd/ceph-11.2.0/obj-x86_64-linux-gnu/src/pybind/rados/pyre
>> >> > x/r
>> >> > ados.c
>> >> > :4439)
>> >> >   File "rados.pyx", line 2657, in rados.Ioctx.execute
>> >> > (/tmp/buildd/ceph-11.2.0/obj-x86_64-linux-gnu/src/pybind/rados/pyre
>> >> > x/r
>> >> > ados.c
>> >> > :34056)
>> >> > rados.PermissionError: Ioctx.read(rbd): failed to read test
>> >> >
>> >
>
--
To unsubscribe from this list: send the line "unsubscribe ceph-devel" in
the body of a message to majordomo@xxxxxxxxxxxxxxx
More majordomo info at  http://vger.kernel.org/majordomo-info.html




[Index of Archives]     [CEPH Users]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]
  Powered by Linux