On 04/04/2014 09:18 AM, Veli-Pekka Kestilä wrote: > On 3.4.2014 23:41, Adrian Sevcenco wrote: >> On 04/02/2014 09:54 PM, Bill Oliver wrote: >>> Just to see if I can do it, I thought I'd set up a small grid/cluster >>> using fedora. Does anybody know of a good step-by-step guide for this >>> (preferably free and online :-) )? >> As for the answer to your initial inquiry i would highly recommend ROCKS >> Clusters that is based on CENTOS. it will automatically install and >> manage your nodes from a single point (the FrontEnd server)(with NFS >> shared home). >> > I have tried ROCKS before (granted it was maybe 6 years ago) and it > seems good if you have very lage number of nodes. If not then I would > recommend setting the things yourself. Most programs run in scientific that would sum to: set up common authentication, common logging, monitoring, nfs shared home (optional but very useful), the resource manager and individual management of settings and packages of each worker node. But this is up to the admin preferences (i use a personal cluster with 4 nodes (32 cores) and from my point of view it saves me a lot of time instead administering each node by hand) (it takes 10 minutes to re-install all nodes with different settings and new updated packages) (for the 40 nodes grid cluster it takes the same 10 minutes ;) ) > clusters use MPI for shared communication so you will just need MPI i dont know what is meaning of "most" .. in my world of high energy physics we use only distributed computing (so no MPI) > setup I would not waste time job management and such as long as there is > only one person running on the cluster. you might (i am sure you will) be hit by a lots of problems .. mpi does not have a knowledge of nodes and their capacities nor about their availability and status. also if you need to start multiple parallel jobs (eg. you have a 32 cores cluster. a job (on all 32 cores) will take 16 hours to complete. if you launch in the morning at 8 am, the job will will end at 0:00 so until the next morning the cluster will just sit idle) you will need a resource manager. > Setting up maybe 10 to 20 machines with kickstart using fedora with > openmpi, ssh-keys for easy access to nodes and nfs sharing form frontend > shouldn't be that big job. You will take same amount time learning ROCKS > and it will have lots of features you don't need. this is dependent of the previous experiences .. i use rocks without problems .. but i am using it for almost 10 years... YMMV > Also if you don't use the machines all the time and don't have the > hardware yet it might be beneficial to use AWS nodes to do the > calculations. In big HPC setups the real cost of the cluster usually > comes from electricity consumption and cooling so you should factor > these costs in when deciding what to do. I cannot agree more! the advent of cheap IaaS clouds brings a lot of useful opportunities :) Adrian
Attachment:
smime.p7s
Description: S/MIME Cryptographic Signature
-- users mailing list users@xxxxxxxxxxxxxxxxxxxxxxx To unsubscribe or change subscription options: https://admin.fedoraproject.org/mailman/listinfo/users Fedora Code of Conduct: http://fedoraproject.org/code-of-conduct Guidelines: http://fedoraproject.org/wiki/Mailing_list_guidelines Have a question? Ask away: http://ask.fedoraproject.org
