I’m still persevering with this, if anyone can assist, I would truly appreciate it. As I said previously, I’ve been able to identify that the error is specifically a Permission Denied error. I’m dropping the custom keyring in via the CEPH_ARGS environment variable, I’ve also tried using the environment to include the id/name/user to tie it to my monitoring account. I’ve also tried coding those id / name / user into the ceph.conf file with no effect either. I’ve been trying to dump the state of the LibCephFS object to identify what exactly is going on; although I can't see a way to dump everything (that would be far too easy), I can call the conf_get command with a key to return the state of individual settings. I can do this with ‘keyring’ and it returns the location I specified in the environment, similarly, I can request mon_host or fsid, and those return their values from the configuration file, however, I cannot return id, name, user, or client.id, client.name, or client.user, although I don't know for sure if those are the correct setting names I should be requesting? So, it seems that whatever mechanism I use to try to populate the user, I cannot yet get the library to honour it. Does anyone have any further clues that might help me resolve this? Many Thanks, Adam. -----Original Message----- From: Adam Carrgilson (NBI) <Adam.Carrgilson@xxxxxxxxx> Sent: 27 July 2022 09:51 To: ceph-users@xxxxxxx Subject: Re: LibCephFS Python Mount Failure It feels like that, but I have specified --id client.monitoring inside of the environment variable together with the --keyring definition. Is there anyway to query the library so that I can see all the variables that it thinks are active and debug from there? Many Thanks, Adam. From: Gregory Farnum <gfarnum@xxxxxxxxxx> Sent: 26 July 2022 16:41 To: Adam Carrgilson (NBI) <Adam.Carrgilson@xxxxxxxxx> Cc: ceph-users@xxxxxxx Subject: Re: Re: LibCephFS Python Mount Failure It looks like you’re setting environment variables that force your new keyring, it you aren’t telling the library to use your new CephX user. So it opens your new keyring and looks for the default (client.admin) user and doesn’t get anything. -Greg On Tue, Jul 26, 2022 at 7:54 AM Adam Carrgilson (NBI) <Adam.Carrgilson@xxxxxxxxx<mailto:Adam.Carrgilson@xxxxxxxxx>> wrote: I've disabled the part of the script that catches the Python exception and allowed it to print everything out and it looks like the OSError with the code 13, is a permissions error: Traceback (most recent call last): File "./get-ceph-quota-statistics.py", line 274, in <module> main(args) File "./get-ceph-quota-statistics.py", line 30, in main cephfs = login() # holds CephFS bindings File "./get-ceph-quota-statistics.py", line 94, in login cephfs.mount(filesystem_name=b'cephfs') File "cephfs.pyx", line 684, in cephfs.LibCephFS.mount File "cephfs.pyx", line 676, in cephfs.LibCephFS.init cephfs.OSError: error calling ceph_init: Permission denied [Errno 13] Now I've tested a FUSE mount with the same keyfile and that functions as expected, so I'm having to assume that somehow the Python script either doesn't have all of the properties I've supplied (which I doubt, because I can point it at files with admin credentials and it works fine), something within the Python CephFS library might be hardcoded to particular values which I'm having problems with, or maybe something else? Is there a way to interrogate the Python object before I do the cephfs.mount, just to confirm the options are as I expect? Alternatively, python-cephfs wraps around the CephFS library, right? Does the CephFS FUSE component utilise the same CephFS library? If not, is there a way to call something else on the command line directly to rule out problems there? Many Thanks, Adam. -----Original Message----- From: Adam Carrgilson (NBI) <Adam.Carrgilson@xxxxxxxxx<mailto:Adam.Carrgilson@xxxxxxxxx>> Sent: 25 July 2022 16:24 To: ceph-users@xxxxxxx<mailto:ceph-users@xxxxxxx> Cc: Bogdan Adrian Velica <vbogdan@xxxxxxxxx<mailto:vbogdan@xxxxxxxxx>> Subject: Re: LibCephFS Python Mount Failure Thanks Bogdan, I’m running this script at the moment as my development system’s root user account, I don’t have a particular ceph user on this standalone system, and I don’t think I’ll be able to control the user account of the monitoring hosts either (I think they might run under a user account dedicated to the monitoring) but I’m interested to what you think I should test here? I can definitely run the code as the root user, it can read my custom configuration and key files, when I specify those providing the admin user credentials, it works as expected, but when I specify the monitoring credentials it errors with that ceph_init message. My script can open and read the configuration and key files (I print those to screen), and I do attempt to pull back the environment before I execute the mount, and it does include my addition of the CEPH_ARGS. That said, those also work when those particular files are for the admin ceph account, and that can’t be picking up anything from the default locations as I’ve deliberately removed them from there. Is there any way to make the Python LibCephFS more verbose to better understand its error message? Adam. From: Bogdan Adrian Velica <vbogdan@xxxxxxxxx<mailto:vbogdan@xxxxxxxxx>> Sent: 25 July 2022 14:50 To: Adam Carrgilson (NBI) <Adam.Carrgilson@xxxxxxxxx<mailto:Adam.Carrgilson@xxxxxxxxx>> Cc: ceph-users@xxxxxxx<mailto:ceph-users@xxxxxxx> Subject: Re: LibCephFS Python Mount Failure Hi Adam, I think this might be related to the user you are running the script as, try running the scrip as ceph user (or the user you are running your ceph with). Also make sure the variable os.environ.get is used (i might be mistaking here). do a print or something first to see the key is loaded. Just my 2 cents... Best of luck, -- Bogdan Velica Ceph Support Engineer croit GmbH, Freseniusstr. 31h, 81247 Munich<https://www.google.com/maps/search/Freseniusstr.+31h,+81247+Munich?entry=gmail&source=g> Com. register: Amtsgericht Munich HRB 231263 Web: https://croit.io<https://croit.io/> On Mon, Jul 25, 2022 at 4:06 PM Adam Carrgilson (NBI) <Adam.Carrgilson@xxxxxxxxx<mailto:Adam.Carrgilson@xxxxxxxxx><mailto:Adam.Carrgilson@xxxxxxxxx<mailto:Adam.Carrgilson@xxxxxxxxx>>> wrote: Hi all, I'm trying to put together a script to gather CephFS quota utilisation. I'm using the CephFS Python library from here: https://docs.ceph.com/en/latest/cephfs/api/libcephfs-py/ and I've followed the rather a good guide on how to use it here: https://jayjeetc.medium.com/up-and-running-with-libcephfs-7629455f0cdc#934a I have been able to get this working, however; I want this to be able to be portable to run it on our monitoring agents, and specifically, I want to be able to use a limited permission account, so read-only permissions and network limitations. I originally couldn't find a method to specify a custom keyfile to use through the library, but with some assistance, I've found that I can use the Python command: os.environ["CEPH_ARGS"] = "--keyring=/path/to/keyring" to provide the library with that which works great (with my admin account). And therein lies the error, I've created an account just for this use case, but when I provide the keyfile with those credentials, I get the response: OSError(13, 'error calling ceph_init') I can give the limited permission account capabilities matching the admin account, but it continues to fail in the same way, pointing to it not being a permissions issue. Is there something obvious that I've done wrong, or an alternative method that might be a better approach, how can I get this to function for my monitoring? I've installed python-cephfs through my systems package manager, it's version 14.2.22, and I'm connecting through to a Nautilus system which is likewise version 14.2.22. Many Thanks, Adam. _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |