Hi again,
We see these type of errors on all our glusterfs servers : Failed to get fd context for a non-anonymous fd, file: (null), gfid
I even took some fresh VM, build from scratch and installed glusterfs 4.1.6 (2 node setup). I also connected 4 clients and loaded some data ( around
600 Mb with random files). Within a couple of hours the same errors in the logs.
Should we simply ignore these errors?
Thanks,
Max
Van: Graaf, Max de
Verzonden: vrijdag 14 december 2018 08:27
Aan: 'Raghavendra Gowdappa'
CC: Gluster-users
Onderwerp: RE: [Gluster-users] Failed to get fd context for a non-anonymous fd
Hi,
So messages like “: Failed to get fd context for a non-anonymous fd, file: (null), gfid: 2aed88da-0664-4070-9202-b856eb1e3c96” we can
simply ignore?
Looking at the logs after the updrage this one keeps coming back:
grep null /var/log/glusterfs/bricks/apps-glusterfs-gst.log
2018-12-12 16:16:20.664304] E [MSGID: 113040] [posix-helpers.c:1671:__posix_fd_ctx_get] 0-gst-posix: Failed to get fd context for a non-anonymous fd, file: (null), gfid: d497d8e0-7246-4c97-beed-8d927a2242f1
[2018-12-12 20:25:20.793225] E [MSGID: 113040] [posix-helpers.c:1671:__posix_fd_ctx_get] 0-gst-posix: Failed to get fd context for a non-anonymous fd, file: (null), gfid: d497d8e0-7246-4c97-beed-8d927a2242f1
[2018-12-12 21:10:20.503040] E [MSGID: 113040] [posix-helpers.c:1671:__posix_fd_ctx_get] 0-gst-posix: Failed to get fd context for a non-anonymous fd, file: (null), gfid: d497d8e0-7246-4c97-beed-8d927a2242f1
[2018-12-13 20:41:19.462145] E [MSGID: 113040] [posix-helpers.c:1671:__posix_fd_ctx_get] 0-gst-posix: Failed to get fd context for a non-anonymous fd, file: (null), gfid: d497d8e0-7246-4c97-beed-8d927a2242f1
[2018-12-13 21:34:20.088353] E [MSGID: 113040] [posix-helpers.c:1671:__posix_fd_ctx_get] 0-gst-posix: Failed to get fd context for a non-anonymous fd, file: (null), gfid: d497d8e0-7246-4c97-beed-8d927a2242f1
[2018-12-13 21:56:19.957970] E [MSGID: 113040] [posix-helpers.c:1671:__posix_fd_ctx_get] 0-gst-posix: Failed to get fd context for a non-anonymous fd, file: (null), gfid: d497d8e0-7246-4c97-beed-8d927a2242f1
[2018-12-13 23:35:19.859388] E [MSGID: 113040] [posix-helpers.c:1671:__posix_fd_ctx_get] 0-gst-posix: Failed to get fd context for a non-anonymous fd, file: (null), gfid: d497d8e0-7246-4c97-beed-8d927a2242f1
[2018-12-14 00:45:20.160086] E [MSGID: 113040] [posix-helpers.c:1671:__posix_fd_ctx_get] 0-gst-posix: Failed to get fd context for a non-anonymous fd, file: (null), gfid: d497d8e0-7246-4c97-beed-8d927a2242f1
Tried to find the file with
gfid: d497d8e0-7246-4c97-beed-8d927a2242f1 but it looks like it doesn’t exist.
Thanks,
Max
Van: Raghavendra Gowdappa [mailto:rgowdapp@xxxxxxxxxx]
Verzonden: vrijdag 14 december 2018 07:32
Aan: Graaf, Max de
CC: Gluster-users
Onderwerp: Re: [Gluster-users] Failed to get fd context for a non-anonymous fd
Hi,
We updated our glusterfs servers and clients from 3.10 to 4.1.6. Although all seems to be working just fine we see errors like these on almost all glusterfs servers:
[2018-12-14 02:35:03.524378] I [addr.c:55:compare_addr_and_update] 0-/apps/glusterfs-gst: allowed = "*", received addr = "10.159.240.195"
[2018-12-14 02:35:03.524410] I [login.c:111:gf_auth] 0-auth/login: allowed user names: 9d79471c-ce22-4e48-ba0d-81f297ed5ba1
[2018-12-14 02:35:03.524419] I [MSGID: 115029] [server-handshake.c:495:server_setvolume] 0-gst-server: accepted client from CTX_ID:37e79003-38db-48bf-8408-f7e9feaaa44c-GRAPH_ID:0-PID:4012-HOST:GRPACCAALCGST01-PC_NAME:gst-client-1-RECON_NO:-0
(version: 4.1.6)
[2018-12-14 02:35:03.572494] I [MSGID: 115036] [server.c:483:server_rpc_notify] 0-gst-server: disconnecting connection from CTX_ID:37e79003-38db-48bf-8408-f7e9feaaa44c-GRAPH_ID:0-PID:4012-HOST:GRPACCAALCGST01-PC_NAME:gst-client-1-RECON_NO:-0
[2018-12-14 02:35:03.572549] I [MSGID: 115013] [server-helpers.c:286:do_fd_cleanup] 0-gst-server: fd cleanup on <gfid:2aed88da-0664-4070-9202-b856eb1e3c96>
A cleanup was initiated for this fd as client got disconnected.
[2018-12-14 02:35:03.572697] I [socket.c:3844:socket_submit_reply] 0-tcp.gst-server: not connected (priv->connected = -1)
[2018-12-14 02:35:03.572718] E [rpcsvc.c:1378:rpcsvc_submit_generic] 0-rpc-service: failed to submit message (XID: 0x18, Program: GlusterFS 3.3, ProgVers: 330,
Proc: 42) to rpc-transport (tcp.gst-server)
[2018-12-14 02:35:03.572807] E [server.c:137:server_submit_reply] (-->/lib64/libgfrpc.so.0(+0x817e) [0x7f022c85717e] -->/usr/lib64/glusterfs/4.1.6/xlator/protocol/server.so(+0x335b9)
[0x7f02179665b9] -->/usr/lib64/glusterfs/4.1.6/xlator/protocol/server.so(+0xae2e) [0x7f021793de2e] ) 0-: Reply submission failed
[2018-12-14 02:35:03.572823] E [server.c:144:server_submit_reply] (-->/lib64/libgfrpc.so.0(+0x817e) [0x7f022c85717e] -->/usr/lib64/glusterfs/4.1.6/xlator/protocol/server.so(+0x335b9)
[0x7f02179665b9] -->/usr/lib64/glusterfs/4.1.6/xlator/protocol/server.so(+0xae96) [0x7f021793de96] ) 0-: Reply submission failed
[2018-12-14 02:35:03.572877] E [MSGID: 113040] [posix-helpers.c:1671:__posix_fd_ctx_get] 0-gst-posix: Failed to get fd context for a non-anonymous fd, file: (null),
gfid: 2aed88da-0664-4070-9202-b856eb1e3c96
This, most likely is an operation racing with the cleanup initiated. Since this condition is triggered by a disconnect, this looks to be benign, unless you've seen any functional issues at application layer.
[2018-12-14 02:35:03.572889] W [MSGID: 113006] [posix-inode-fd-ops.c:2018:posix_flush] 0-gst-posix: pfd is NULL on fd=0x7f01e80092c8 [Invalid argument]
[2018-12-14 02:35:03.573034] I [MSGID: 101055] [client_t.c:444:gf_client_unref] 0-gst-server: Shutting down connection CTX_ID:37e79003-38db-48bf-8408-f7e9feaaa44c-GRAPH_ID:0-PID:4012-HOST:GRPACCAALCGST01-PC_NAME:gst-client-1-RECON_NO:-0
[2018-12-14 02:35:33.750415] I [addr.c:55:compare_addr_and_update] 0-/apps/glusterfs-gst: allowed = "*", received addr = "10.159.240.227"
[2018-12-14 02:35:33.750439] I [login.c:111:gf_auth] 0-auth/login: allowed user names: 9d79471c-ce22-4e48-ba0d-81f297ed5ba1
[2018-12-14 02:35:33.750447] I [MSGID: 115029] [server-handshake.c:495:server_setvolume] 0-gst-server: accepted client from CTX_ID:e76cbd89-2f9e-404e-887a-256783d2fc1f-GRAPH_ID:0-PID:5385-HOST:GRPACCAPDCGST01-PC_NAME:gst-client-1-RECON_NO:-0
(version: 4.1.6)
[2018-12-14 02:35:33.817381] I [MSGID: 115036] [server.c:483:server_rpc_notify] 0-gst-server: disconnecting connection from CTX_ID:e76cbd89-2f9e-404e-887a-256783d2fc1f-GRAPH_ID:0-PID:5385-HOST:GRPACCAPDCGST01-PC_NAME:gst-client-1-RECON_NO:-0
[2018-12-14 02:35:33.818343] I [MSGID: 101055] [client_t.c:444:gf_client_unref] 0-gst-server: Shutting down connection CTX_ID:e76cbd89-2f9e-404e-887a-256783d2fc1f-GRAPH_ID:0-PID:5385-HOST:GRPACCAPDCGST01-PC_NAME:gst-client-1-RECON_NO:-0
We have 3 separate environments and all 3 have a similar setup with 2 glusterfs servers with 6 to 10 glusterfs clients connected. No heavy load at all, all running
on CentOS 7.1 en 7.3.
Any thoughts?
Thanks,
Max
_______________________________________________
Gluster-users mailing list
Gluster-users@xxxxxxxxxxx
https://lists.gluster.org/mailman/listinfo/gluster-users
