Thanks for the clarification, and Rook does use Kubernetes facilities to handle the log collection so it sounds like we're good to go. On 9/27/17, 9:45 AM, "John Spray" <jspray@xxxxxxxxxx> wrote: >On Wed, Sep 27, 2017 at 5:36 PM, Travis Nielsen ><Travis.Nielsen@xxxxxxxxxxx> wrote: >> To expand on the scenario, I'm working in a Kubernetes environment where >> the MDS instances are somewhat ephemeral. If an instance (pod) dies or >>the >> machine is restarted, Kubernetes will start a new one in its place. To >> handle the failed pod scenario, I'd appreciate if you could help me >> understand MDS better. >> >> 1) MDS instances are stateless, correct? If so, I'm assuming when an MDS >> instance dies, a new MDS instance (with a new ID) can be brought up and >> assigned its rank without any side effects other than disruption during >> the failover. Or is there a reason to treat them more like mons that >>need >> to survive reboots and maintain state? > >Yep, completely stateless. Don't forget logs though -- for ephemeral >instances, it would be a good idea to have them sending their logs >somewhere central, so that we don't lose all the history whenever a >container restarts (you may very well have already covered this in >general in the context of Rook). > >> 2) Will there be any side effects from MDS instances being somewhat >> ephemeral? For example, if a new instance came up every hour or every >>day, >> what challenges would I run into besides cleaning up the old cephx keys? > >While switching daemons around is an online operation, it is not >without some impact to client IOs, and the freshly started MDS daemon >will generally have a less well populated cache than the one it is >replacing. > >John > >> >> Thanks! >> Travis >> >> >> >> >> On 9/27/17, 3:01 AM, "John Spray" <jspray@xxxxxxxxxx> wrote: >> >>>On Wed, Sep 27, 2017 at 12:09 AM, Travis Nielsen >>><Travis.Nielsen@xxxxxxxxxxx> wrote: >>>> Is it possible to use the same cephx key for all instances of MDS or >>>>do >>>> they each require their own? Mons require the same keyring so I tried >>>> following the same pattern by creating a keyring with "mds.", but the >>>>MDS >>>> is complaining about not being authorized when it tries to start. Am I >>>> missing something or is this not possible for MDS keys? If I create a >>>> unique key for each MDS instance it works fine, but it would simplify >>>>my >>>> scenario if I could use the same key. I'm running on Luminous. >>> >>>I've never heard of anyone trying to do this. >>> >>>It's probably not a great idea, because if all MDS daemons are using >>>the same key then you lose the ability to simply remove an MDS's key >>>to ensure that it can't talk to the system any more. This is useful >>>when tearing something down, because it means you're not taking it on >>>faith that the daemon is really physically stopped. >>> >>>John >>> >>>> The key was generated with this: >>>> ceph auth get-or-create-key mds. osd allow * mds allow mon allow >>>>profile >>>> mds >>>> >>>> >>>> >>>> The keyring contents are: >>>> [mds.] >>>> key = AQD62spZw3zRGhAAkHHVokP3BDf8PEy4+vXGMg== >>>> >>>> >>>> I run the following with that keyring: >>>> ceph-mds --foreground --name=mds.mymds -i mymds >>>> >>>> And I see the error: >>>> 2017-09-26 22:55:55.973047 7fb004459200 -1 mds.mds81c2n ERROR: failed >>>>to >>>> authenticate: (22) Invalid argument >>>> >>>> >>>> >>>> Thanks, >>>> Travis >>>> >>>> >>>> -- >>>> To unsubscribe from this list: send the line "unsubscribe ceph-devel" >>>>in >>>> the body of a message to majordomo@xxxxxxxxxxxxxxx >>>> More majordomo info at >>>>https://na01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fvger.ke >>>>rn >>>>el.org%2Fmajordomo-info.html&data=02%7C01%7CTravis.Nielsen%40quantum.co >>>>m% >>>>7C00d1db42478d48fa8c6508d5058ec254%7C322a135f14fb4d72aede122272134ae0%7 >>>>C1 >>>>%7C0%7C636421033061815149&sdata=3Vu79xeZbnb1jwhGE85PACq6qByVE6vUlPjp8pj >>>>rv >>>>hA%3D&reserved=0 >> -- To unsubscribe from this list: send the line "unsubscribe ceph-devel" in the body of a message to majordomo@xxxxxxxxxxxxxxx More majordomo info at http://vger.kernel.org/majordomo-info.html
