Hello Greg, I saw that the site of the previous link of the logs uses a very short expiring time so I uploaded it to another one: http://www.mediafire.com/download/gikiy7cqs42cllt/ceph-mds.th1-mon001.log.tar.gz Thanks, Jasper ________________________________________ Van: gregory.farnum@xxxxxxxxxxx [gregory.farnum@xxxxxxxxxxx] namens Gregory Farnum [gfarnum@xxxxxxxxxx] Verzonden: donderdag 30 oktober 2014 1:03 Aan: Jasper Siero CC: John Spray; ceph-users Onderwerp: Re: mds isn't working anymore after osd's running full On Wed, Oct 29, 2014 at 7:51 AM, Jasper Siero <jasper.siero@xxxxxxxxxxxxxxxxx> wrote: > Hello Greg, > > I added the debug options which you mentioned and started the process again: > > [root@th1-mon001 ~]# /usr/bin/ceph-mds -i th1-mon001 --pid-file /var/run/ceph/mds.th1-mon001.pid -c /etc/ceph/ceph.conf --cluster ceph --reset-journal 0 > old journal was 9483323613~134233517 > new journal start will be 9621733376 (4176246 bytes past old end) > writing journal head > writing EResetJournal entry > done > [root@th1-mon001 ~]# /usr/bin/ceph-mds -i th1-mon001 -c /etc/ceph/ceph.conf --cluster ceph --undump-journal 0 journaldumptgho-mon001 > undump journaldumptgho-mon001 > start 9483323613 len 134213311 > writing header 200.00000000 > writing 9483323613~1048576 > writing 9484372189~1048576 > writing 9485420765~1048576 > writing 9486469341~1048576 > writing 9487517917~1048576 > writing 9488566493~1048576 > writing 9489615069~1048576 > writing 9490663645~1048576 > writing 9491712221~1048576 > writing 9492760797~1048576 > writing 9493809373~1048576 > writing 9494857949~1048576 > writing 9495906525~1048576 > writing 9496955101~1048576 > writing 9498003677~1048576 > writing 9499052253~1048576 > writing 9500100829~1048576 > writing 9501149405~1048576 > writing 9502197981~1048576 > writing 9503246557~1048576 > writing 9504295133~1048576 > writing 9505343709~1048576 > writing 9506392285~1048576 > writing 9507440861~1048576 > writing 9508489437~1048576 > writing 9509538013~1048576 > writing 9510586589~1048576 > writing 9511635165~1048576 > writing 9512683741~1048576 > writing 9513732317~1048576 > writing 9514780893~1048576 > writing 9515829469~1048576 > writing 9516878045~1048576 > writing 9517926621~1048576 > writing 9518975197~1048576 > writing 9520023773~1048576 > writing 9521072349~1048576 > writing 9522120925~1048576 > writing 9523169501~1048576 > writing 9524218077~1048576 > writing 9525266653~1048576 > writing 9526315229~1048576 > writing 9527363805~1048576 > writing 9528412381~1048576 > writing 9529460957~1048576 > writing 9530509533~1048576 > writing 9531558109~1048576 > writing 9532606685~1048576 > writing 9533655261~1048576 > writing 9534703837~1048576 > writing 9535752413~1048576 > writing 9536800989~1048576 > writing 9537849565~1048576 > writing 9538898141~1048576 > writing 9539946717~1048576 > writing 9540995293~1048576 > writing 9542043869~1048576 > writing 9543092445~1048576 > writing 9544141021~1048576 > writing 9545189597~1048576 > writing 9546238173~1048576 > writing 9547286749~1048576 > writing 9548335325~1048576 > writing 9549383901~1048576 > writing 9550432477~1048576 > writing 9551481053~1048576 > writing 9552529629~1048576 > writing 9553578205~1048576 > writing 9554626781~1048576 > writing 9555675357~1048576 > writing 9556723933~1048576 > writing 9557772509~1048576 > writing 9558821085~1048576 > writing 9559869661~1048576 > writing 9560918237~1048576 > writing 9561966813~1048576 > writing 9563015389~1048576 > writing 9564063965~1048576 > writing 9565112541~1048576 > writing 9566161117~1048576 > writing 9567209693~1048576 > writing 9568258269~1048576 > writing 9569306845~1048576 > writing 9570355421~1048576 > writing 9571403997~1048576 > writing 9572452573~1048576 > writing 9573501149~1048576 > writing 9574549725~1048576 > writing 9575598301~1048576 > writing 9576646877~1048576 > writing 9577695453~1048576 > writing 9578744029~1048576 > writing 9579792605~1048576 > writing 9580841181~1048576 > writing 9581889757~1048576 > writing 9582938333~1048576 > writing 9583986909~1048576 > writing 9585035485~1048576 > writing 9586084061~1048576 > writing 9587132637~1048576 > writing 9588181213~1048576 > writing 9589229789~1048576 > writing 9590278365~1048576 > writing 9591326941~1048576 > writing 9592375517~1048576 > writing 9593424093~1048576 > writing 9594472669~1048576 > writing 9595521245~1048576 > writing 9596569821~1048576 > writing 9597618397~1048576 > writing 9598666973~1048576 > writing 9599715549~1048576 > writing 9600764125~1048576 > writing 9601812701~1048576 > writing 9602861277~1048576 > writing 9603909853~1048576 > writing 9604958429~1048576 > writing 9606007005~1048576 > writing 9607055581~1048576 > writing 9608104157~1048576 > writing 9609152733~1048576 > writing 9610201309~1048576 > writing 9611249885~1048576 > writing 9612298461~1048576 > writing 9613347037~1048576 > writing 9614395613~1048576 > writing 9615444189~1048576 > writing 9616492765~1044159 > done. > [root@th1-mon001 ~]# service ceph start mds > === mds.th1-mon001 === > Starting Ceph mds.th1-mon001 on th1-mon001... > starting mds.th1-mon001 at :/0 > > > The new logs: > http://pastebin.com/wqqjuEpy These don't have the increased debugging levels set. :( I'm not sure where you could have put them that they didn't get picked up, but make sure it's in the ceph.conf that this mds daemon is referring to. (You can see the debug levels in use in the "--- logging levels ---" section; they appear to all be default.) -Greg _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
 |