Re: Recovery question

Peter Hinman <Peter.Hinman@xxxxxxxx> · Wed, 29 Jul 2015 14:46:02 -0600



    Hi Greg - 

    
    So at the moment, I seem to be trying to resolve a permission
    error.  

     === osd.3 === 

       Mounting xfs on stor-2:/var/lib/ceph/osd/ceph-3

       2015-07-29 13:35:08.809536 7f0a0262e700  0 librados: osd.3
      authentication error (1) Operation not permitted

       Error connecting to cluster: PermissionError

       failed: 'timeout 30 /usr/bin/ceph -c /etc/ceph/ceph.conf
      --name=osd.3 --keyring=/var/lib/ceph/osd/ceph-3/keyring osd crush
      create-or-move -- 3 3.64 host=stor-2 root=default'

       ceph-disk: Error: ceph osd start failed: Command
      '['/usr/sbin/service', 'ceph', '--cluster', 'ceph', 'start',
      'osd.3']' returned non-zero exit status 1

       ceph-disk: Error: One or more partitions failed to activate
    

    Is there a way to identify the cause of this PermissionError?  I've
    copied the client.bootstrap-osd key from the output of ceph auth
    list, and pasted it into /var/lib/ceph/bootstrap-osd/ceph.keyring,
    but that has not resolve the error.

    
    But it sounds like you are saying that even once I get this
    resolved, I have no hope of recovering the data?

    -- 
Peter Hinman


    On 7/29/2015 1:57 PM, Gregory Farnum
      wrote:

    
      This sounds like you're trying to reconstruct a cluster after
        destroying the monitors. That is...not going to work well. The
        monitors define the cluster and you can't move OSDs into
        different clusters. We have ideas for how to reconstruct
        monitors and it can be done manually with a lot of hassle, but
        the process isn't written down and there aren't really fools I
        help with it. :/

        -Greg
      

        On Wed, Jul 29, 2015 at 5:48 PM Peter Hinman <Peter.Hinman@xxxxxxxx>
          wrote:

        
        I've got a
          situation that seems on the surface like it should be

          recoverable, but I'm struggling to understand how to do it.

          
          I had a cluster of 3 monitors, 3 osd disks, and 3 journal
          ssds. After

          multiple hardware failures, I pulled the 3 osd disks and 3
          journal ssds

          and am attempting to bring them back up again on new hardware
          in a new

          cluster.  I see plenty of documentation on how to zap and
          initialize and

          add "new" osds, but I don't see anything on rebuilding with
          existing osd

          disks.

          
          Could somebody provide guidance on how to do this?  I'm
          running 94.2 on

          all machines.

          
          Thanks,

          
          --

          Peter Hinman

          
          _______________________________________________

          ceph-users mailing list

          ceph-users@xxxxxxxxxxxxxx

          http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com

        
_______________________________________________
ceph-users mailing list
ceph-users@xxxxxxxxxxxxxx
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com