Re: devops/clusterSSH - (offtopic)

[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

 



Yo Rick!!!

You are da mann!!! Owe you a six pac of OJ one of these days!

I completely forgot about the changing IP on the remote client, means
that while the ssh key is valid, the request for confirmation will
occur for the "known_host" (iirc). So yeah I'll have to mod either the
local or remote sshd/ssh config file for that..

As to the initial IP of the cloned droplets, I populate predata with
that via the API that's used when generating the set of remote
instances.

I'll take a look at the tools you mentioned as well..

Thanks much for all, for being another set of "eyes"!!

Really do appreciate the comments.!!


On Tue, Nov 15, 2016 at 12:36 PM, Rick Stevens <ricks@xxxxxxxxxxxxxx> wrote:
> On 11/15/2016 07:39 AM, bruce wrote:
>> Hey!
>>
>> --This is offtopic from fed --
>>
>> Thanks to all who've commented on the SSH/screen issue! Think I got
>> that part resolved.
>>
>> Now, in order to use the ipAddresses that have been generated for the
>> droplets, need to get opinions/thoughts on apps to use to handle
>> connecting to the different numbers of droplets/instances for the
>> project.
>>
>> The following items are a kind of rough overview of what we're
>> thinking needs to be done.
>>
>> We're looking at clusterSSH (even though it appears to need to be
>> built from source!). Other thoughts/opinions on apps that might
>> suffice would be good.
>
> ClusterSSH is available from the standard repos, no need to build it:
>
> [root@prophead ~]# dnf list clusterssh*
> Last metadata expiration check: 2:18:20 ago on Tue Nov 15 06:48:02 2016.
> Installed Packages
> clusterssh.noarch               4.08-1.fc24             @updates
>
>> Any tool we use, will need to initially handle ~50 droplets/instances,
>> and scale to ~500-1000
>>
>>
>> The tool should:
>> - be able to fire up group(s) of servers based on ipAddress (type of
>> droplet -fetch/parse/etc..)
>> -be able to generate cmd to single/multiple
>> -be able to display either single/group of terms based on ipAddress
>> (single/group)
>> -be able to group the term(ipAddress) -- single term/multiple term in
>> the group(s)
>> -be able to switch between the groups, which in turn display the terms
>> for the group
>
> cssh can do all of that, but once launched in interactive mode, you
> can't "switch between the groups". If you use the /etc/clusters
> mechanism, e.g. "cssh group-from-cluster-file", then the terminals for
> all machines in that cluster will be displayed on your workspace. You
> could have multiple workspaces and run a cssh for a different cluster on
> each one and switch between workspaces. We do that a lot.
>
>> The use case...
>> -the crawler spins up a number of droplets/instances
>> -the crawler generates the required ipAddresses and "groups" the
>> ipAddresses, based on fetch/parse use
>
> Uhm, if the droplets get different IPs each time, one of the first
> things I'd have them do is figure out what their IP is and report it
> to a database or something so it's known by your app.
>
>> -All the "instances' are generated/cloned to have the required apps on
>> the server in order to be fetch or parse - there's no need to
>> upload/scp files to the remote instances
>
> This is outside the spec of a terminal manager.
>
>> Tool Requirements:
>> -Nice if the termManagerApp is able to use config/xternal files to
>> handle the ipAddresses to create groups as required
>> -Nice if the termManagerApp is able to display terms in a given group
>> -App has to handle external pub/priv key, all terms (cloned) have the same key
>
> That's a function of the access method. If the droplets regenerate ssh
> keys and such each time they spin up, you'll have issues since the
> ssh client will see a new host key each time you spin up a droplet.
>
>> The termManagerApp needs to be able to display terms from the selected group
>>
>> TermManagerApp needs to be able to send same command to all terms in
>> the displayed group
>> TermManagerApp needs to be able to select a given term, and insert
>> cmds to that term only
>> Terms being displayed, should display "realtime" window update
>
> cssh can do that.
>
>> Nice if termManagerApp can display 20-50 terms simultaneously
>
> With cssh, this is a function of your screen and font sizes. You'll
> need a big screen and relatively small fonts. Too small for these old
> eyes of mine.
>
>> Basically, the tool/app will be used to allow the project to manage
>> the multiple instances/VMs that are being run for the crawl.
>>
>> Project use functions requires:
>>  -Running commands on remote servers
>>  -Checking on the progress of the remote processes via the screen function
>>  -Starting up/Shutting Down remote servers as needed
>>  -TBD..
>
> You may also want to look at pdsh/dshbak (available from the repos) to
> send commands to remote machines. pdsh doesn't open windows as cssh
> does, but sends commands and receives the responses back. The responses
> are fed to dshbak to sort them into host order so they're not all
> jumbled together. Other tools to look at are ansible and puppet (both
> available from the repos) and there are others I haven't mentioned. All
> of them have their uses and strengths and weaknesses and there really
> isn't any one, single "best" solution. If there were, there wouldn't be
> this plethora of tools. I think you'll find you use several, depending
> on exactly what you're trying to accomplish at that particular time.
>
> Note that the issues with ssh keys and such will rear their heads
> regardless of what tool(s) you use. If you want password-less access,
> each droplet needs to use the same ssh hostkey each time (and ideally
> the same IP) or you'll need to set the /etc/ssh/ssh_config or
> ~/.ssh/config file's "StrictHostKeyChecking" to "no" or "ask" (not
> necessarily a good thing--it depends on your level of paranoia).
> ----------------------------------------------------------------------
> - Rick Stevens, Systems Engineer, AllDigital    ricks@xxxxxxxxxxxxxx -
> - AIM/Skype: therps2        ICQ: 226437340           Yahoo: origrps2 -
> -                                                                    -
> -                  Heisenberg _may_ have slept here                  -
> ----------------------------------------------------------------------
> _______________________________________________
> users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
> To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx
_______________________________________________
users mailing list -- users@xxxxxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to users-leave@xxxxxxxxxxxxxxxxxxxxxxx



[Index of Archives]     [Older Fedora Users]     [Fedora Announce]     [Fedora Package Announce]     [EPEL Announce]     [EPEL Devel]     [Fedora Magazine]     [Fedora Summer Coding]     [Fedora Laptop]     [Fedora Cloud]     [Fedora Advisory Board]     [Fedora Education]     [Fedora Security]     [Fedora Scitech]     [Fedora Robotics]     [Fedora Infrastructure]     [Fedora Websites]     [Anaconda Devel]     [Fedora Devel Java]     [Fedora Desktop]     [Fedora Fonts]     [Fedora Marketing]     [Fedora Management Tools]     [Fedora Mentors]     [Fedora Package Review]     [Fedora R Devel]     [Fedora PHP Devel]     [Kickstart]     [Fedora Music]     [Fedora Packaging]     [Fedora SELinux]     [Fedora Legal]     [Fedora Kernel]     [Fedora OCaml]     [Coolkey]     [Virtualization Tools]     [ET Management Tools]     [Yum Users]     [Yosemite News]     [Gnome Users]     [KDE Users]     [Fedora Art]     [Fedora Docs]     [Fedora Sparc]     [Libvirt Users]     [Fedora ARM]

  Powered by Linux