Hello, The past couple of weeks I had some issues with firmware on the OS hard drives in my gluster cluster. I have recently fixed the issue, and am bringing my bricks back into the volume. I am running gluster 3.7.6 and am running into the following issue: When I add the brick and rebalance, the operation fails after a couple minutes. The errors I find in the rebalance log is this: [2017-06-05 13:38:40.441671] E [MSGID: 109010] [dht-rebalance.c:2259:gf_defrag_get_entry] 0-hpcscratch-dht: /LV_Fitting/code/C gfid not present [2017-06-05 13:38:40.450341] E [MSGID: 109010] [dht-rebalance.c:2259:gf_defrag_get_entry] 0-hpcscratch-dht: /LV_Fitting/code/C/NoCov_NoImm gfid not present [2017-06-05 13:38:40.450380] E [MSGID: 109010] [dht-rebalance.c:2259:gf_defrag_get_entry] 0-hpcscratch-dht: /LV_Fitting/code/C/simulate gfid not present [2017-06-05 13:38:40.459365] E [MSGID: 109010] [dht-rebalance.c:2259:gf_defrag_get_entry] 0-hpcscratch-dht: /LV_Fitting/code/C/NoCov_NoImm/fits_generate gfid not present [2017-06-05 13:38:40.468756] E [MSGID: 109010] [dht-rebalance.c:2259:gf_defrag_get_entry] 0-hpcscratch-dht: /LV_Fitting/code/C/NoCov_NoImm/fits_generate/N_0_vector.dat gfid not present [2017-06-05 13:38:40.495645] E [MSGID: 109010] [dht-rebalance.c:2259:gf_defrag_get_entry] 0-hpcscratch-dht: LV_Fitting/code/C/simulate/RK45_Integrate.c gfid not present [2017-06-05 13:38:40.512336] E [MSGID: 109010] [dht-rebalance.c:2259:gf_defrag_get_entry] 0-hpcscratch-dht: /LV_Fitting/output/line_search gfid not present [2017-06-05 13:38:40.512373] E [MSGID: 109010] [dht-rebalance.c:2259:gf_defrag_get_entry] 0-hpcscratch-dht: /LV_Fitting/output/mcmc gfid not present [2017-06-05 13:38:40.517808] E [dht-rebalance.c:2992:gf_defrag_fix_layout] 0-hpcscratch-dht: Setxattr failed for /LV_Fitting/output/line_search [2017-06-05 13:38:40.518025] E [MSGID: 109016] [dht-rebalance.c:3006:gf_defrag_fix_layout] 0-hpcscratch-dht: Fix layout failed for /LV_Fitting/output [2017-06-05 13:38:40.518136] E [MSGID: 109016] [dht-rebalance.c:3006:gf_defrag_fix_layout] 0-hpcscratch-dht: Fix layout failed for /LV_Fitting There are about 102,000 of the gfid error, but only a few errors for the fix layout failed. Is there any way to recover from this issue? -- Mike Jarsulic Sr. HPC Administrator Center for Research Informatics | University of Chicago 773.702.2066 _______________________________________________ Gluster-users mailing list Gluster-users@xxxxxxxxxxx http://lists.gluster.org/mailman/listinfo/gluster-users