On 12/20/21 12:47, Igor Fedotov wrote:
Thanks for the info.
Just in case - is write caching disabled for the disk in question?
What's the output for "hdparm -W </path-to-disk-dev>" ?
no, it is enabled. Shall I disable that on all OSDs?
And am I right that you have got a non-default (false) setting for
osd_fast_shutdown config parameter? Any other non-defaults for OSD
config parameters?
osd_fast_shutdown I have set to true after few OSDs failed to start, but
it seems it did not have much of an effect.
the other settings:
[root@lcst0001 ~]# ceph config dump|grep osd
global advanced osd_pool_default_pg_autoscale_mode off
mon advanced mon_max_pg_per_osd 1000
mgr advanced osd_deep_scrub_interval
4838400.000000
osd advanced bluefs_buffered_io true
osd advanced objecter_inflight_ops 10240
osd advanced osd_deep_scrub_interval 4838400.000000
osd advanced osd_fast_shutdown false
osd advanced osd_max_backfills 32
osd advanced osd_max_pg_per_osd_hard_ratio 20.000000
osd advanced osd_max_scrubs 1
osd advanced osd_op_num_threads_per_shard_ssd 4
*
osd advanced osd_op_thread_timeout 90
osd advanced osd_pool_default_ec_fast_read true
osd advanced osd_recovery_max_active 16
osd advanced osd_recovery_max_active_hdd 64
osd advanced osd_recovery_sleep_hdd 0.000000
osd advanced osd_scrub_auto_repair true
Best regards,
Andrej
Thanks,
Igor
On 12/20/2021 1:13 PM, Andrej Filipcic wrote:
On 12/20/21 10:47, Igor Fedotov wrote:
On 12/20/2021 12:26 PM, Andrej Filipcic wrote:
On 12/20/21 10:09, Igor Fedotov wrote:
Hi Andrej,
3) Please set debug-bluefs to 20, retry the OSD start and share
the log.
http://www-f9.ijs.si/~andrej/ceph-osd.611.log-20211220-short.gz
http://www-f9.ijs.si/~andrej/ceph-osd.611-debug20.log
sorry, bad copy/paste
Andrej
This is exactly the same link as the original one - it doesn't have
verbose bluefs logging...
Igor Fedotov
Ceph Lead Developer
Looking for help with your Ceph cluster? Contact us at https://croit.io
croit GmbH, Freseniusstr. 31h, 81247 Munich
CEO: Martin Verges - VAT-ID: DE310638492
Com. register: Amtsgericht Munich HRB 231263
Web: https://croit.io | YouTube: https://goo.gl/PGE1Bx
--
_____________________________________________________________
prof. dr. Andrej Filipcic, E-mail:Andrej.Filipcic@xxxxxx
Department of Experimental High Energy Physics - F9
Jozef Stefan Institute, Jamova 39, P.o.Box 3000
SI-1001 Ljubljana, Slovenia
Tel.: +386-1-477-3674 Fax: +386-1-477-3166
-------------------------------------------------------------
--
Igor Fedotov
Ceph Lead Developer
Looking for help with your Ceph cluster? Contact us athttps://croit.io
croit GmbH, Freseniusstr. 31h, 81247 Munich
CEO: Martin Verges - VAT-ID: DE310638492
Com. register: Amtsgericht Munich HRB 231263
Web:https://croit.io | YouTube:https://goo.gl/PGE1Bx
--
_____________________________________________________________
prof. dr. Andrej Filipcic, E-mail:Andrej.Filipcic@xxxxxx
Department of Experimental High Energy Physics - F9
Jozef Stefan Institute, Jamova 39, P.o.Box 3000
SI-1001 Ljubljana, Slovenia
Tel.: +386-1-477-3674 Fax: +386-1-477-3166
-------------------------------------------------------------
_______________________________________________
ceph-users mailing list -- ceph-users@xxxxxxx
To unsubscribe send an email to ceph-users-leave@xxxxxxx
