Re: Cephadm bootstrap sets incorrect user & group id for ceph user

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Ah I thought they were fixed. Still weird, the IDs inside the container definitely were not the same as on the host.
I think adding a check would be quite nice. Not sure if I have the skills for that but I don't mind giving it a try.
Should it check if the IDs on the host and container match?

On Thu, 26 Aug 2021 at 14:00, Sebastian Wagner <sewagner@xxxxxxxxxx> wrote:

Hi Massim,

cephadm looks for the IDs in the container:

https://github.com/ceph/ceph/blob/master/src/cephadm/cephadm#L4499

so as long as the host's (in this case the VM) IDs stay the same thorough the lifecycle, everything works. This seems to be an edge case for me. Would you be interested in adding a check to cephadm? Maybe to


https://github.com/ceph/ceph/blob/b7b73d5daa020643d0d4b1bba0d6fb6b091b005b/src/cephadm/cephadm#L5798


?

Am 26.08.21 um 13:33 schrieb Massim Knaapen:
Hi,

I got this cluster setup on 3VMs for testing purposes. Every time one of the boxes rebooted the OSDs daemons failed.
They worked fine, as long as I didn't reboot....

Then I noticed that the UID and GID for the Ceph user were set incorrectly.
If I understand correctly: the `ceph) user needs to have a UID and GID set to 167.
In the Docker container this was correct, but on the host it was a different story.

The fix was quite easy of course:
$ usermod -u 167 ceph
$ groupmod -u 167 ceph

But the errors are not that clear (only found some reference about this in the OSD service log, see below).

If Ceph needs to have this user set to this specific ID. Maybe it would be nice to add this to the Troubleshooting section of the docs?

This is the output of the OSD service log (in case someone tries to search for this error):
`Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.133+0000 7f025ef4a080  0 set uid:gid to 167:167 (ceph:ceph)                                                                            
Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.133+0000 7f025ef4a080  0 ceph version 16.2.5 (0883bdea7337b95e4b611c768c0279868462204a) pacific (stable), process ceph-osd, pid 8      
Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.133+0000 7f025ef4a080  0 pidfile_write: ignore empty --pid-file                                                                        
Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.137+0000 7f025ef4a080  1 bdev(0x5646bc068800 /var/lib/ceph/osd/ceph-0/block) open path /var/lib/ceph/osd/ceph-0/block                  
Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.137+0000 7f025ef4a080  1 bdev(0x5646bc068800 /var/lib/ceph/osd/ceph-0/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size>
Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.137+0000 7f025ef4a080  1 bluestore(/var/lib/ceph/osd/ceph-0) _set_cache_sizes cache_size 1073741824 meta 0.45 kv 0.45 data 0.06        
Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.137+0000 7f025ef4a080  1 bdev(0x5646bc068c00 /var/lib/ceph/osd/ceph-0/block) open path /var/lib/ceph/osd/ceph-0/block                  
Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.137+0000 7f025ef4a080  1 bdev(0x5646bc068c00 /var/lib/ceph/osd/ceph-0/block) open size 214744170496 (0x31ffc00000, 200 GiB) block_size>
Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.137+0000 7f025ef4a080  1 bluefs add_block_device bdev 1 path /var/lib/ceph/osd/ceph-0/block size 200 GiB                           
Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.137+0000 7f025ef4a080  1 bdev(0x5646bc068c00 /var/lib/ceph/osd/ceph-0/block) close                                                    
Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.161+0000 7f025ef4a080  1 bdev(0x5646bc068800 /var/lib/ceph/osd/ceph-0/block) close                                                    
Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.425+0000 7f025ef4a080  0 starting osd.0 osd_data /var/lib/ceph/osd/ceph-0 /var/lib/ceph/osd/ceph-0/journal                            
Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.425+0000 7f025ef4a080 -1 Falling back to public interface                                                                              
Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.429+0000 7f025ef4a080  0 load: jerasure load: lrc load: isa                                                                            
Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.437+0000 7f025ef4a080  1 bdev(0x5646bc03c400 /var/lib/ceph/osd/ceph-0/block) open path /var/lib/ceph/osd/ceph-0/block                  
Aug 24 17:00:09 alpha0001 bash[7301]: debug 2021-08-24T17:00:09.437+0000 7f025ef4a080 -1 bdev(0x5646bc03c400 /var/lib/ceph/osd/ceph-0/block) open open got: (13) Permission denied  `

_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx
_______________________________________________
Dev mailing list -- dev@xxxxxxx
To unsubscribe send an email to dev-leave@xxxxxxx

[Index of Archives]     [CEPH Users]     [Ceph Devel]     [Ceph Large]     [Information on CEPH]     [Linux BTRFS]     [Linux USB Devel]     [Video for Linux]     [Linux Audio Users]     [Yosemite News]     [Linux Kernel]     [Linux SCSI]

  Powered by Linux