Emmanuel, You probably missed the "daemon" keyword after the "ceph" command name. Here's the docs for pacific: https://docs.ceph.com/en/pacific/cephfs/troubleshooting/ So, your command should've been: # ceph daemon mds.icadmin011 dump cache /tmp/dump.txt You could also dump the ops in flight with: # ceph daemon mds.icadmin011 dump_ops_in_flight On Wed, May 24, 2023 at 1:38 PM Emmanuel Jaep <emmanuel.jaep@xxxxxxxxx> wrote: > Hi, > > we are running a cephfs cluster with the following version: > ceph version 16.2.10 (45fa1a083152e41a408d15505f594ec5f1b4fe17) pacific > (stable) > > Several MDSs are reporting slow requests: > HEALTH_WARN 4 MDSs report slow requests > [WRN] MDS_SLOW_REQUEST: 4 MDSs report slow requests > mds.icadmin011(mds.5): 1 slow requests are blocked > 30 secs > mds.icadmin015(mds.6): 2 slow requests are blocked > 30 secs > mds.icadmin006(mds.4): 8 slow requests are blocked > 30 secs > mds.icadmin007(mds.2): 2 slow requests are blocked > 30 secs > > According to Quincy's documentation ( > https://docs.ceph.com/en/quincy/cephfs/troubleshooting/), this can be > investigated by issuing: > ceph mds.icadmin011 dump cache /tmp/dump.txt > > Unfortunately, this command fails: > no valid command found; 10 closest matches: > pg stat > pg getmap > pg dump [all|summary|sum|delta|pools|osds|pgs|pgs_brief...] > pg dump_json [all|summary|sum|pools|osds|pgs...] > pg dump_pools_json > pg ls-by-pool <poolstr> [<states>...] > pg ls-by-primary <id|osd.id> [<pool:int>] [<states>...] > pg ls-by-osd <id|osd.id> [<pool:int>] [<states>...] > pg ls [<pool:int>] [<states>...] > pg dump_stuck [inactive|unclean|stale|undersized|degraded...] > [<threshold:int>] > Error EINVAL: invalid command > > > I imagine that it is related to the fact that we are running the Pacific > version and not the Quincy version. > > When looking at the Pacific's documentation ( > https://docs.ceph.com/en/pacific/cephfs/health-messages/), I should: > > Use the ops admin socket command to list outstanding metadata operations. > > Unfortunately, I fail to really understand what I'm supposed to do. Can > someone give a pointer? > > Best, > > Emmanuel > _______________________________________________ > ceph-users mailing list -- ceph-users@xxxxxxx > To unsubscribe send an email to ceph-users-leave@xxxxxxx > > -- Milind _______________________________________________ ceph-users mailing list -- ceph-users@xxxxxxx To unsubscribe send an email to ceph-users-leave@xxxxxxx
 |