We have a directory of debian package files under cephfs and over the past week or so we've noticed this issue when trying to read some of the packages from that directory. Here are some messages from the system logs, they all look like they come from the same place in the code. [Thu Apr 20 17:22:34 2017] libceph: get_reply osd11 tid 31399 data 2441216 > preallocated 1720320, skipping [Mon Apr 24 11:54:09 2017] libceph: get_reply osd42 tid 150169 data 2083124 > preallocated 1048576, skipping (Also, its Ubuntu 14.04.4, not 16.04.1, but the kernel is 4.9.10 and ceph is 10.2.5) On Mon, Apr 24, 2017 at 2:48 PM, Ilya Dryomov <idryomov@xxxxxxxxx> wrote: > On Mon, Apr 24, 2017 at 5:51 PM, Wyllys Ingersoll > <wyllys.ingersoll@xxxxxxxxxxxxxx> wrote: >> Ubuntu 16.04.1 >> Ceph 10.2.5 >> Linux 4.9.10 kernel >> >> We are having problems accessing some files in cephfs, the process >> reading the files become hung when trying to read them (md5sum, for >> example). When these hangups occur, there are messages in the system >> logs that look like this: >> >> "libceph: get_reply osd6 tid 55 data 1305852 > preallocated 1048576, skipping" >> >> The cluster shows healthy status and all OSD are reporting as up/up. >> >> Wondering if our hanging read issues are related to the messages being >> reported, they seem to be since we only see the messages once we >> encounter the issue reading the files. > > Definitely -- this message is printed when the reply is dropped on the > floor. Can you tell us more about these files and/or provide a sample > of these messages? > > Thanks, > > Ilya -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
