There's a little bit of python called ceph-create-keys, which is invoked by the upstart scripts. You can kill the running processes, and edit them out of the scripts, without direct harm. (Their purpose is to create some standard keys which the newer deployment tools rely on to do things like create OSDs, etc.) -Greg Software Engineer #42 @ http://inktank.com | http://ceph.com On Thu, Apr 18, 2013 at 3:20 PM, Matthew Roy <imjustmatthew@xxxxxxxxx> wrote: > On 04/18/2013 06:03 PM, Joao Eduardo Luis wrote: >> >> There's definitely some command messages being forwarded, but AFAICT >> they're being forwarded to the monitor, not by the monitor, which by >> itself is a good omen towards the monitor being the leader :-) >> >> In any case, nothing in the trace's code path indicates we could be a >> peon, unless the monitor itself believed to be the leader. If you take >> a closer look, you'll see that we come from 'handle_last()', which is >> bound to happen only on the leader (we'll assert otherwise). For the >> monitor to be receiving these messages it must mean the peons believe >> him to be the leader -- or we have so many bugs going around that it's >> just madness! >> >> In all seriousness, when I was chasing after this bug, Matthew sent me >> his logs with higher debug levels -- no craziness going around :-) >> >> -Joao >> > > Is there a way to tell who's being "denied"? Even if it's just log > pollution I'd like to know which client is misconfigured. There are > similar messages in all the mon logs: > > mon.a: > 2013-04-18 18:16:51.254378 7fc7c6d10700 1 -- > [2001:470:8:dd9::20]:6789/0 --> [2001:470:8:dd9::21]:6789/0 -- > route(mon_command_ack([auth,get-or-create,client.admin,mon,allow > *,osd,allow *,mds,allow]=-13 access denied v775211) v1 tid 8867608) v2 > -- ?+0 0x7fc61a18b160 con 0x253f700 > > > mon.b: > 2013-04-18 18:16:49.670758 7f37c7afa700 20 -- > [2001:470:8:dd9::21]:6789/0 >> [2001:470:8:dd9::21]:0/22372 > pipe(0x7f383c070b70 sd=90 :6789 s=2 pgs=1 cs=1 l=1).writer encoding 7 > 0x7f37f49876a0 > mon_command_ack([auth,get-or-create,client.admin,mon,allow *,osd,allow > *,mds,allow]=-13 access denied v775209) v1 > > (mon.c was removed since the first log file in the thread) > > mon.d: > 2013-04-18 18:16:51.304897 7f927d40f700 1 -- > [2001:470:8:dd9:7271:bcff:febd:e398]:6789/0 --> client.? > [2001:470:8:dd9::21]:0/26333 -- > mon_command_ack([auth,get-or-create,client.admin,mon,allow *,osd,allow > *,mds,allow]=-13 access denied v775211) v1 -- ?+0 0x7f923c0230a0 > > The spacing on these messages is about 0.001s so there's a lot of them > going around. All these systems are running 0.60-472-g327002e > > Matthew > > > > > -- > Matthew _______________________________________________ ceph-users mailing list ceph-users@xxxxxxxxxxxxxx http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com