Yes I did that as well. Node1 of my cluster doesn't show (using clust) vm service whereas others do. I guess the problem is with the rgmanager but it looks to be running fine. Paras. On Tue, Oct 6, 2009 at 12:59 PM, Daniela Anzellotti <daniela.anzellotti@xxxxxxxxxxxxx> wrote: > Hi Paras, > > did you reboot all the cluster nodes? I needed a complete reboot (actually I > was angry enough to switch everything off and on again): a restart of the > cluster suite was not enough. > > As far as I understood, since the cluster was not able to bring a VM in a > good running state, it decided that the VM has to migrate to another node... > and at the end all the VMs was trying to migrate from one node to another > with the result that I had all VMs starting on all the cluster nodes. > Restarting the cluster suite didn't kill a lot of processes that was > stubbornly trying to migrate virtual machines... > > Daniela > > Paras pradhan wrote: >> >> Adding use_virsh=0 works great. Now I do not have my vm starting at >> all the nodes. This is a good fix. Thanks .. >> >> The only problem left is I do not see rgmanager running on my node1 >> and the clustat of node2 and node3 is reporting vm as migrating >> >> o/p >> >> Service Name Owner (Last) >> State >> ------- ---- ----- ------ >> ----- >> vm:guest1 cvtst1 >> migrating >> [root@cvtst3 vms]# >> >> >> Thanks >> Paras. >> >> On Tue, Oct 6, 2009 at 10:48 AM, Daniela Anzellotti >> <daniela.anzellotti@xxxxxxxxxxxxx> wrote: >>> >>> Hi Paras, >>> >>> yes. At least it looks so... >>> >>> We have a cluster of two nodes + a quorum disk (it's not configured as a >>> "two-node cluster") >>> >>> They are running Scientific Linux 5.x, kernel 2.6.18-128.7.1.el5xen and >>> >>> openais-0.80.6-8.el5.x86_64 >>> cman-2.0.115-1.el5.x86_64 >>> rgmanager-2.0.52-1.el5.x86_64 >>> >>> The XEN VMs access the disk as simple block devices. >>> Disks are on a SAN, configured with Clustered LVM. >>> >>> xen-3.0.3-94.el5_4.1.x86_64 >>> xen-libs-3.0.3-94.el5_4.1.x86_64 >>> >>> VM configuration files are as the following >>> >>> name = "www1" >>> uuid = "3bd3e910-23c0-97ee-55ab-086260ef1e53" >>> memory = 1024 >>> maxmem = 1024 >>> vcpus = 1 >>> bootloader = "/usr/bin/pygrub" >>> vfb = [ "type=vnc,vncunused=1,keymap=en-us" ] >>> disk = [ "phy:/dev/vg_cluster/www1.disk,xvda,w", \ >>> "phy:/dev/vg_cluster/www1.swap,xvdb,w" ] >>> vif = [ "mac=00:16:3e:da:00:07,bridge=xenbr1" ] >>> on_poweroff = "destroy" >>> on_reboot = "restart" >>> on_crash = "restart" >>> extra = "xencons=tty0 console=tty0" >>> >>> >>> I changed in /etc/cluster/cluster.conf all the VM directive from >>> >>> <vm autostart="1" domain="rhcs1_dom" exclusive="0" \ >>> migrate="live" name="www1" path="/etc/xen" recovery="restart"/> >>> >>> to >>> >>> <vm autostart="1" use_virsh="0" domain="rhcs1_dom" exclusive="0" \ >>> migrate="live" name="www1" path="/etc/xen" recovery="restart"/> >>> >>> >>> Rebooted the cluster nodes and it started working again... >>> >>> As i said I hope I'll not have any other bad surprise (I tested a VM >>> migration and it is working too), but at least cluster it's working now >>> (it >>> was not able to start a VM, before)! >>> >>> Ciao >>> Daniela >>> >>> >>> Paras pradhan wrote: >>>> >>>> So you mean your cluster is running fine with the CMAN >>>> cman-2.0.115-1.el5.x86_64 ? >>>> >>>> Which version of openais are you running? >>>> >>>> Thanks >>>> Paras. >>>> >>>> >>>> On Mon, Oct 5, 2009 at 7:19 AM, Daniela Anzellotti >>>> <daniela.anzellotti@xxxxxxxxxxxxx> wrote: >>>>> >>>>> Hi all, >>>>> >>>>> I had a problem similar to Paras's one today: yum updated the following >>>>> rpms >>>>> last week and today (I had to restart the cluster) the cluster was not >>>>> able >>>>> to start vm: services. >>>>> >>>>> Oct 02 05:31:05 Updated: openais-0.80.6-8.el5.x86_64 >>>>> Oct 02 05:31:07 Updated: cman-2.0.115-1.el5.x86_64 >>>>> Oct 02 05:31:10 Updated: rgmanager-2.0.52-1.el5.x86_64 >>>>> >>>>> Oct 03 04:03:12 Updated: xen-libs-3.0.3-94.el5_4.1.x86_64 >>>>> Oct 03 04:03:12 Updated: xen-libs-3.0.3-94.el5_4.1.i386 >>>>> Oct 03 04:03:16 Updated: xen-3.0.3-94.el5_4.1.x86_64 >>>>> >>>>> >>>>> So, after checked the vm.sh script, I add the declaration use_virsh="0" >>>>> in >>>>> the VM definition in the cluster.conf (as suggested by Brem, thanks!) >>>>> and >>>>> everything is now working again. >>>>> >>>>> >>>>> BTW I didn't understand if the problem was caused by the new XEN >>>>> version >>>>> or >>>>> the new openais one, thus I disabled automatic updates for both. >>>>> >>>>> I hope I'll not have any other bad surprise... >>>>> >>>>> Thank you, >>>>> cheers, >>>>> Daniela >>>>> >>>>> >>>>> Paras pradhan wrote: >>>>>> >>>>>> Yes this is very strange. I don't know what to do now. May be re >>>>>> create the cluster? But not a good solution actually. >>>>>> >>>>>> Packages : >>>>>> >>>>>> Kernel: kernel-xen-2.6.18-164.el5 >>>>>> OS: Full updated of CentOS 5.3 except CMAN downgraded to >>>>>> cman-2.0.98-1.el5 >>>>>> >>>>>> Other packages related to cluster suite: >>>>>> >>>>>> rgmanager-2.0.52-1.el5.centos >>>>>> cman-2.0.98-1.el5 >>>>>> xen-3.0.3-80.el5_3.3 >>>>>> xen-libs-3.0.3-80.el5_3.3 >>>>>> kmod-gfs-xen-0.1.31-3.el5_3.1 >>>>>> kmod-gfs-xen-0.1.31-3.el5_3.1 >>>>>> kmod-gfs-0.1.31-3.el5_3.1 >>>>>> gfs-utils-0.1.18-1.el5 >>>>>> gfs2-utils-0.1.62-1.el5 >>>>>> lvm2-2.02.40-6.el5 >>>>>> lvm2-cluster-2.02.40-7.el5 >>>>>> openais-0.80.3-22.el5_3.9 >>>>>> >>>>>> Thanks! >>>>>> Paras. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> On Wed, Sep 30, 2009 at 10:02 AM, brem belguebli >>>>>> <brem.belguebli@xxxxxxxxx> wrote: >>>>>>> >>>>>>> Hi Paras, >>>>>>> >>>>>>> Your cluster.conf file seems correct. If it is not a ntp issue, I >>>>>>> don't see anything except a bug that causes this, or some >>>>>>> prerequisite >>>>>>> that is not respected. >>>>>>> >>>>>>> May be you could post the versions (os, kernel, packages etc...) you >>>>>>> are using, someone may have hit the same issue with your versions. >>>>>>> >>>>>>> Brem >>>>>>> >>>>>>> 2009/9/30, Paras pradhan <pradhanparas@xxxxxxxxx>: >>>>>>>> >>>>>>>> All of the nodes are synced with ntp server. So this is not the case >>>>>>>> with me. >>>>>>>> >>>>>>>> Thanks >>>>>>>> Paras. >>>>>>>> >>>>>>>> On Tue, Sep 29, 2009 at 6:29 PM, Johannes Rußek >>>>>>>> <johannes.russek@xxxxxxxxxxxxxxxxx> wrote: >>>>>>>>> >>>>>>>>> make sure the time on the nodes is in sync, apparently when a node >>>>>>>>> has >>>>>>>>> too >>>>>>>>> much offset, you won't see rgmanager (even though the process is >>>>>>>>> running). >>>>>>>>> this happened today and setting the time fixed it for me. afaicr >>>>>>>>> there >>>>>>>>> was >>>>>>>>> no sign of this in the logs though. >>>>>>>>> johannes >>>>>>>>> >>>>>>>>> Paras pradhan schrieb: >>>>>>>>>> >>>>>>>>>> I don't see rgmanager . >>>>>>>>>> >>>>>>>>>> Here is the o/p from clustat >>>>>>>>>> >>>>>>>>>> [root@cvtst1 cluster]# clustat >>>>>>>>>> Cluster Status for test @ Tue Sep 29 15:53:33 2009 >>>>>>>>>> Member Status: Quorate >>>>>>>>>> >>>>>>>>>> Member Name >>>>>>>>>> ID >>>>>>>>>> Status >>>>>>>>>> ------ ---- >>>>>>>>>> ---- >>>>>>>>>> ------ >>>>>>>>>> cvtst2 1 >>>>>>>>>> Online >>>>>>>>>> cvtst1 2 >>>>>>>>>> Online, >>>>>>>>>> Local >>>>>>>>>> cvtst3 3 >>>>>>>>>> Online >>>>>>>>>> >>>>>>>>>> >>>>>>>>>> Thanks >>>>>>>>>> Paras. >>>>>>>>>> >>>>>>>>>> On Tue, Sep 29, 2009 at 3:44 PM, brem belguebli >>>>>>>>>> <brem.belguebli@xxxxxxxxx> wrote: >>>>>>>>>> >>>>>>>>>>> It looks correct, rgmanager seems to start on all nodes >>>>>>>>>>> >>>>>>>>>>> what gives you clustat ? >>>>>>>>>>> >>>>>>>>>>> If rgmanager doesn't show, check out the logs something may have >>>>>>>>>>> gone >>>>>>>>>>> wrong. >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> 2009/9/29 Paras pradhan <pradhanparas@xxxxxxxxx>: >>>>>>>>>>> >>>>>>>>>>>> Change to 7 and i got this log >>>>>>>>>>>> >>>>>>>>>>>> Sep 29 15:33:50 cvtst1 rgmanager: [23295]: <notice> Shutting >>>>>>>>>>>> down >>>>>>>>>>>> Cluster Service Manager... >>>>>>>>>>>> Sep 29 15:33:50 cvtst1 clurgmgrd[22869]: <notice> Shutting down >>>>>>>>>>>> Sep 29 15:33:50 cvtst1 clurgmgrd[22869]: <notice> Shutting down >>>>>>>>>>>> Sep 29 15:33:50 cvtst1 clurgmgrd[22869]: <notice> Shutdown >>>>>>>>>>>> complete, >>>>>>>>>>>> exiting >>>>>>>>>>>> Sep 29 15:33:50 cvtst1 rgmanager: [23295]: <notice> Cluster >>>>>>>>>>>> Service >>>>>>>>>>>> Manager is stopped. >>>>>>>>>>>> Sep 29 15:33:51 cvtst1 clurgmgrd[23324]: <notice> Resource Group >>>>>>>>>>>> Manager Starting >>>>>>>>>>>> Sep 29 15:33:51 cvtst1 clurgmgrd[23324]: <info> Loading Service >>>>>>>>>>>> Data >>>>>>>>>>>> Sep 29 15:33:51 cvtst1 clurgmgrd[23324]: <debug> Loading >>>>>>>>>>>> Resource >>>>>>>>>>>> Rules >>>>>>>>>>>> Sep 29 15:33:52 cvtst1 clurgmgrd[23324]: <debug> 21 rules loaded >>>>>>>>>>>> Sep 29 15:33:52 cvtst1 clurgmgrd[23324]: <debug> Building >>>>>>>>>>>> Resource >>>>>>>>>>>> Trees >>>>>>>>>>>> Sep 29 15:33:52 cvtst1 clurgmgrd[23324]: <debug> 0 resources >>>>>>>>>>>> defined >>>>>>>>>>>> Sep 29 15:33:52 cvtst1 clurgmgrd[23324]: <debug> Loading >>>>>>>>>>>> Failover >>>>>>>>>>>> Domains >>>>>>>>>>>> Sep 29 15:33:52 cvtst1 clurgmgrd[23324]: <debug> 1 domains >>>>>>>>>>>> defined >>>>>>>>>>>> Sep 29 15:33:52 cvtst1 clurgmgrd[23324]: <debug> 1 events >>>>>>>>>>>> defined >>>>>>>>>>>> Sep 29 15:33:52 cvtst1 clurgmgrd[23324]: <info> Initializing >>>>>>>>>>>> Services >>>>>>>>>>>> Sep 29 15:33:52 cvtst1 clurgmgrd[23324]: <info> Services >>>>>>>>>>>> Initialized >>>>>>>>>>>> Sep 29 15:33:52 cvtst1 clurgmgrd[23324]: <debug> Event: Port >>>>>>>>>>>> Opened >>>>>>>>>>>> Sep 29 15:33:52 cvtst1 clurgmgrd[23324]: <info> State change: >>>>>>>>>>>> Local >>>>>>>>>>>> UP >>>>>>>>>>>> Sep 29 15:33:52 cvtst1 clurgmgrd[23324]: <info> State change: >>>>>>>>>>>> cvtst2 >>>>>>>>>>>> UP >>>>>>>>>>>> Sep 29 15:33:52 cvtst1 clurgmgrd[23324]: <info> State change: >>>>>>>>>>>> cvtst3 >>>>>>>>>>>> UP >>>>>>>>>>>> Sep 29 15:33:57 cvtst1 clurgmgrd[23324]: <debug> Event (1:2:1) >>>>>>>>>>>> Processed >>>>>>>>>>>> Sep 29 15:33:57 cvtst1 clurgmgrd[23324]: <debug> Event (0:1:1) >>>>>>>>>>>> Processed >>>>>>>>>>>> Sep 29 15:33:57 cvtst1 clurgmgrd[23324]: <debug> Event (0:3:1) >>>>>>>>>>>> Processed >>>>>>>>>>>> Sep 29 15:34:02 cvtst1 clurgmgrd[23324]: <debug> 3 events >>>>>>>>>>>> processed >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>>> Anything unusual here? >>>>>>>>>>>> >>>>>>>>>>>> Paras. >>>>>>>>>>>> >>>>>>>>>>>> On Tue, Sep 29, 2009 at 11:51 AM, brem belguebli >>>>>>>>>>>> <brem.belguebli@xxxxxxxxx> wrote: >>>>>>>>>>>> >>>>>>>>>>>>> I use log_level=7 to have more debugging info. >>>>>>>>>>>>> >>>>>>>>>>>>> It seems 4 is not enough. >>>>>>>>>>>>> >>>>>>>>>>>>> Brem >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>>> 2009/9/29, Paras pradhan <pradhanparas@xxxxxxxxx>: >>>>>>>>>>>>> >>>>>>>>>>>>>> Withe log_level of 3 I got only this >>>>>>>>>>>>>> >>>>>>>>>>>>>> Sep 29 10:31:31 cvtst1 rgmanager: [7170]: <notice> Shutting >>>>>>>>>>>>>> down >>>>>>>>>>>>>> Cluster Service Manager... >>>>>>>>>>>>>> Sep 29 10:31:31 cvtst1 clurgmgrd[6673]: <notice> Shutting down >>>>>>>>>>>>>> Sep 29 10:31:41 cvtst1 clurgmgrd[6673]: <notice> Shutdown >>>>>>>>>>>>>> complete, >>>>>>>>>>>>>> exiting >>>>>>>>>>>>>> Sep 29 10:31:41 cvtst1 rgmanager: [7170]: <notice> Cluster >>>>>>>>>>>>>> Service >>>>>>>>>>>>>> Manager is stopped. >>>>>>>>>>>>>> Sep 29 10:31:42 cvtst1 clurgmgrd[7224]: <notice> Resource >>>>>>>>>>>>>> Group >>>>>>>>>>>>>> Manager Starting >>>>>>>>>>>>>> Sep 29 10:39:06 cvtst1 rgmanager: [10327]: <notice> Shutting >>>>>>>>>>>>>> down >>>>>>>>>>>>>> Cluster Service Manager... >>>>>>>>>>>>>> Sep 29 10:39:16 cvtst1 rgmanager: [10327]: <notice> Cluster >>>>>>>>>>>>>> Service >>>>>>>>>>>>>> Manager is stopped. >>>>>>>>>>>>>> Sep 29 10:39:16 cvtst1 clurgmgrd[10380]: <notice> Resource >>>>>>>>>>>>>> Group >>>>>>>>>>>>>> Manager Starting >>>>>>>>>>>>>> Sep 29 10:39:52 cvtst1 clurgmgrd[10380]: <notice> Member 1 >>>>>>>>>>>>>> shutting >>>>>>>>>>>>>> down >>>>>>>>>>>>>> >>>>>>>>>>>>>> I do not know what the last line means. >>>>>>>>>>>>>> >>>>>>>>>>>>>> rgmanager version I am running is: >>>>>>>>>>>>>> rgmanager-2.0.52-1.el5.centos >>>>>>>>>>>>>> >>>>>>>>>>>>>> I don't what has gone wrong. >>>>>>>>>>>>>> >>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>> Paras. >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>>> On Mon, Sep 28, 2009 at 6:41 PM, brem belguebli >>>>>>>>>>>>>> <brem.belguebli@xxxxxxxxx> wrote: >>>>>>>>>>>>>> >>>>>>>>>>>>>>> you mean it stopped successfully on all the nodes but it is >>>>>>>>>>>>>>> failing >>>>>>>>>>>>>>> to >>>>>>>>>>>>>>> start only on node cvtst1 ? >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> look at the following page to make rgmanager more verbose. >>>>>>>>>>>>>>> It >>>>>>>>>>>>>>> 'll >>>>>>>>>>>>>>> help debug.... >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> http://sources.redhat.com/cluster/wiki/RGManager >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> at Logging Configuration section >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> 2009/9/29 Paras pradhan <pradhanparas@xxxxxxxxx>: >>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Brem, >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> When I try to restart rgmanager on all the nodes, this time >>>>>>>>>>>>>>>> i >>>>>>>>>>>>>>>> do >>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>> see rgmanager running on the first node. But I do see on >>>>>>>>>>>>>>>> other >>>>>>>>>>>>>>>> 2 >>>>>>>>>>>>>>>> nodes. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Log on the first node: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Sep 28 18:13:58 cvtst1 clurgmgrd[24099]: <notice> Resource >>>>>>>>>>>>>>>> Group >>>>>>>>>>>>>>>> Manager Starting >>>>>>>>>>>>>>>> Sep 28 18:17:29 cvtst1 rgmanager: [24627]: <notice> Shutting >>>>>>>>>>>>>>>> down >>>>>>>>>>>>>>>> Cluster Service Manager... >>>>>>>>>>>>>>>> Sep 28 18:17:29 cvtst1 clurgmgrd[24099]: <notice> Shutting >>>>>>>>>>>>>>>> down >>>>>>>>>>>>>>>> Sep 28 18:17:39 cvtst1 clurgmgrd[24099]: <notice> Shutdown >>>>>>>>>>>>>>>> complete, >>>>>>>>>>>>>>>> exiting >>>>>>>>>>>>>>>> Sep 28 18:17:39 cvtst1 rgmanager: [24627]: <notice> Cluster >>>>>>>>>>>>>>>> Service >>>>>>>>>>>>>>>> Manager is stopped. >>>>>>>>>>>>>>>> Sep 28 18:17:40 cvtst1 clurgmgrd[24679]: <notice> Resource >>>>>>>>>>>>>>>> Group >>>>>>>>>>>>>>>> Manager Starting >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> - >>>>>>>>>>>>>>>> It seems service is running , but I do not see rgmanger >>>>>>>>>>>>>>>> running >>>>>>>>>>>>>>>> using clustat >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Don't know what is going on. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>> Paras. >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> On Mon, Sep 28, 2009 at 5:46 PM, brem belguebli >>>>>>>>>>>>>>>> <brem.belguebli@xxxxxxxxx> wrote: >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Paras, >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> Another thing, it would have been more interesting to have >>>>>>>>>>>>>>>>> a >>>>>>>>>>>>>>>>> start >>>>>>>>>>>>>>>>> DEBUG not a stop. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> That's why I was asking you to first stop the vm manually >>>>>>>>>>>>>>>>> on >>>>>>>>>>>>>>>>> all >>>>>>>>>>>>>>>>> your >>>>>>>>>>>>>>>>> nodes, stop eventually rgmanager on all the nodes to reset >>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>> potential wrong states you may have, restart rgmanager. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> If your VM is configured to autostart, this will make it >>>>>>>>>>>>>>>>> start. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> It should normally fail (as it does now). Send out your >>>>>>>>>>>>>>>>> newly >>>>>>>>>>>>>>>>> created >>>>>>>>>>>>>>>>> DEBUG file. >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> 2009/9/29 brem belguebli <brem.belguebli@xxxxxxxxx>: >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Hi Paras, >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I don't know the xen/cluster combination well, but if I do >>>>>>>>>>>>>>>>>> remember >>>>>>>>>>>>>>>>>> well, I think I've read somewhere that when using xen you >>>>>>>>>>>>>>>>>> have >>>>>>>>>>>>>>>>>> to >>>>>>>>>>>>>>>>>> declare the use_virsh=0 key in the VM definition in the >>>>>>>>>>>>>>>>>> cluster.conf. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> This would make rgmanager use xm commands instead of virsh >>>>>>>>>>>>>>>>>> The DEBUG output shows clearly that you are using virsh to >>>>>>>>>>>>>>>>>> manage >>>>>>>>>>>>>>>>>> your >>>>>>>>>>>>>>>>>> VM instead of xm commands. >>>>>>>>>>>>>>>>>> Check out the RH docs about virtualization >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> I'm not a 100% sure about that, I may be completely wrong. >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> Brem >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>> 2009/9/28 Paras pradhan <pradhanparas@xxxxxxxxx>: >>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The only thing I noticed is the message after stopping >>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>> vm >>>>>>>>>>>>>>>>>>> using xm >>>>>>>>>>>>>>>>>>> in all nodes and starting using clusvcadm is >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> "Virtual machine guest1 is blocked" >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> The whole DEBUG file is attached. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>>>>> Paras. >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> On Fri, Sep 25, 2009 at 5:53 PM, brem belguebli >>>>>>>>>>>>>>>>>>> <brem.belguebli@xxxxxxxxx> wrote: >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> There's a problem with the script that is called by >>>>>>>>>>>>>>>>>>>> rgmanager to >>>>>>>>>>>>>>>>>>>> start >>>>>>>>>>>>>>>>>>>> the VM, I don't know what causes it >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> May be you should try something like : >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 1) stop the VM on all nodes with xm commands >>>>>>>>>>>>>>>>>>>> 2) edit the /usr/share/cluster/vm.sh script and add the >>>>>>>>>>>>>>>>>>>> following >>>>>>>>>>>>>>>>>>>> lines (after the #!/bin/bash ): >>>>>>>>>>>>>>>>>>>> exec >/tmp/DEBUG 2>&1 >>>>>>>>>>>>>>>>>>>> set -x >>>>>>>>>>>>>>>>>>>> 3) start the VM with clusvcadm -e vm:guest1 >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> It should fail as it did before. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> edit the the /tmp/DEBUG file and you will be able to see >>>>>>>>>>>>>>>>>>>> where >>>>>>>>>>>>>>>>>>>> it >>>>>>>>>>>>>>>>>>>> fails (it may generate a lot of debug) >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 4) remove the debug lines from /usr/share/cluster/vm.sh >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Post the DEBUG file if you're not able to see where it >>>>>>>>>>>>>>>>>>>> fails. >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> Brem >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> 2009/9/26 Paras pradhan <pradhanparas@xxxxxxxxx>: >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> No I am not manually starting not using automatic init >>>>>>>>>>>>>>>>>>>>> scripts. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I started the vm using: clusvcadm -e vm:guest1 >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> I have just stopped using clusvcadm -s vm:guest1. For >>>>>>>>>>>>>>>>>>>>> few >>>>>>>>>>>>>>>>>>>>> seconds it >>>>>>>>>>>>>>>>>>>>> says guest1 started . But after a while I can see the >>>>>>>>>>>>>>>>>>>>> guest1 on >>>>>>>>>>>>>>>>>>>>> all >>>>>>>>>>>>>>>>>>>>> three nodes. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> clustat says: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Service Name >>>>>>>>>>>>>>>>>>>>> Owner >>>>>>>>>>>>>>>>>>>>> (Last) >>>>>>>>>>>>>>>>>>>>> State >>>>>>>>>>>>>>>>>>>>> ------- ---- >>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>> ------ >>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>> vm:guest1 >>>>>>>>>>>>>>>>>>>>> (none) >>>>>>>>>>>>>>>>>>>>> stopped >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> But I can see the vm from xm li. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> This is what I can see from the log: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Sep 25 17:19:01 cvtst1 clurgmgrd[4298]: <notice> start >>>>>>>>>>>>>>>>>>>>> on >>>>>>>>>>>>>>>>>>>>> vm >>>>>>>>>>>>>>>>>>>>> "guest1" >>>>>>>>>>>>>>>>>>>>> returned 1 (generic error) >>>>>>>>>>>>>>>>>>>>> Sep 25 17:19:01 cvtst1 clurgmgrd[4298]: <warning> #68: >>>>>>>>>>>>>>>>>>>>> Failed >>>>>>>>>>>>>>>>>>>>> to start >>>>>>>>>>>>>>>>>>>>> vm:guest1; return value: 1 >>>>>>>>>>>>>>>>>>>>> Sep 25 17:19:01 cvtst1 clurgmgrd[4298]: <notice> >>>>>>>>>>>>>>>>>>>>> Stopping >>>>>>>>>>>>>>>>>>>>> service vm:guest1 >>>>>>>>>>>>>>>>>>>>> Sep 25 17:19:02 cvtst1 clurgmgrd[4298]: <notice> >>>>>>>>>>>>>>>>>>>>> Service >>>>>>>>>>>>>>>>>>>>> vm:guest1 is >>>>>>>>>>>>>>>>>>>>> recovering >>>>>>>>>>>>>>>>>>>>> Sep 25 17:19:15 cvtst1 clurgmgrd[4298]: <notice> >>>>>>>>>>>>>>>>>>>>> Recovering >>>>>>>>>>>>>>>>>>>>> failed >>>>>>>>>>>>>>>>>>>>> service vm:guest1 >>>>>>>>>>>>>>>>>>>>> Sep 25 17:19:16 cvtst1 clurgmgrd[4298]: <notice> start >>>>>>>>>>>>>>>>>>>>> on >>>>>>>>>>>>>>>>>>>>> vm >>>>>>>>>>>>>>>>>>>>> "guest1" >>>>>>>>>>>>>>>>>>>>> returned 1 (generic error) >>>>>>>>>>>>>>>>>>>>> Sep 25 17:19:16 cvtst1 clurgmgrd[4298]: <warning> #68: >>>>>>>>>>>>>>>>>>>>> Failed >>>>>>>>>>>>>>>>>>>>> to start >>>>>>>>>>>>>>>>>>>>> vm:guest1; return value: 1 >>>>>>>>>>>>>>>>>>>>> Sep 25 17:19:16 cvtst1 clurgmgrd[4298]: <notice> >>>>>>>>>>>>>>>>>>>>> Stopping >>>>>>>>>>>>>>>>>>>>> service vm:guest1 >>>>>>>>>>>>>>>>>>>>> Sep 25 17:19:17 cvtst1 clurgmgrd[4298]: <notice> >>>>>>>>>>>>>>>>>>>>> Service >>>>>>>>>>>>>>>>>>>>> vm:guest1 is >>>>>>>>>>>>>>>>>>>>> recovering >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> Paras. >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> On Fri, Sep 25, 2009 at 5:07 PM, brem belguebli >>>>>>>>>>>>>>>>>>>>> <brem.belguebli@xxxxxxxxx> wrote: >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Have you started your VM via rgmanager (clusvcadm -e >>>>>>>>>>>>>>>>>>>>>> vm:guest1) or >>>>>>>>>>>>>>>>>>>>>> using xm commands out of cluster control (or maybe a >>>>>>>>>>>>>>>>>>>>>> thru >>>>>>>>>>>>>>>>>>>>>> an >>>>>>>>>>>>>>>>>>>>>> automatic init script ?) >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> When clustered, you should never be starting services >>>>>>>>>>>>>>>>>>>>>> (manually or >>>>>>>>>>>>>>>>>>>>>> thru automatic init script) out of cluster control >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> The thing would be to stop your vm on all the nodes >>>>>>>>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>> adequate >>>>>>>>>>>>>>>>>>>>>> xm command (not using xen myself) and try to start it >>>>>>>>>>>>>>>>>>>>>> with >>>>>>>>>>>>>>>>>>>>>> clusvcadm. >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> Then see if it is started on all nodes (send clustat >>>>>>>>>>>>>>>>>>>>>> output) >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> 2009/9/25 Paras pradhan <pradhanparas@xxxxxxxxx>: >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Ok. Please see below. my vm is running on all nodes >>>>>>>>>>>>>>>>>>>>>>> though >>>>>>>>>>>>>>>>>>>>>>> clustat >>>>>>>>>>>>>>>>>>>>>>> says it is stopped. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>> [root@cvtst1 ~]# clustat >>>>>>>>>>>>>>>>>>>>>>> Cluster Status for test @ Fri Sep 25 16:52:34 2009 >>>>>>>>>>>>>>>>>>>>>>> Member Status: Quorate >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Member Name >>>>>>>>>>>>>>>>>>>>>>> ID Status >>>>>>>>>>>>>>>>>>>>>>> ------ ---- >>>>>>>>>>>>>>>>>>>>>>> ---- ------ >>>>>>>>>>>>>>>>>>>>>>> cvtst2 >>>>>>>>>>>>>>>>>>>>>>> 1 >>>>>>>>>>>>>>>>>>>>>>> Online, rgmanager >>>>>>>>>>>>>>>>>>>>>>> cvtst1 >>>>>>>>>>>>>>>>>>>>>>> 2 >>>>>>>>>>>>>>>>>>>>>>> Online, >>>>>>>>>>>>>>>>>>>>>>> Local, rgmanager >>>>>>>>>>>>>>>>>>>>>>> cvtst3 >>>>>>>>>>>>>>>>>>>>>>> 3 >>>>>>>>>>>>>>>>>>>>>>> Online, rgmanager >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Service Name >>>>>>>>>>>>>>>>>>>>>>> Owner (Last) >>>>>>>>>>>>>>>>>>>>>>> State >>>>>>>>>>>>>>>>>>>>>>> ------- ---- >>>>>>>>>>>>>>>>>>>>>>> ----- ------ >>>>>>>>>>>>>>>>>>>>>>> ----- >>>>>>>>>>>>>>>>>>>>>>> vm:guest1 >>>>>>>>>>>>>>>>>>>>>>> (none) >>>>>>>>>>>>>>>>>>>>>>> stopped >>>>>>>>>>>>>>>>>>>>>>> [root@cvtst1 ~]# >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>> o/p of xm li on cvtst1 >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>> [root@cvtst1 ~]# xm li >>>>>>>>>>>>>>>>>>>>>>> Name ID Mem(MiB) >>>>>>>>>>>>>>>>>>>>>>> VCPUs >>>>>>>>>>>>>>>>>>>>>>> State Time(s) >>>>>>>>>>>>>>>>>>>>>>> Domain-0 0 3470 >>>>>>>>>>>>>>>>>>>>>>> 2 >>>>>>>>>>>>>>>>>>>>>>> r----- 28939.4 >>>>>>>>>>>>>>>>>>>>>>> guest1 7 511 >>>>>>>>>>>>>>>>>>>>>>> 1 >>>>>>>>>>>>>>>>>>>>>>> -b---- 7727.8 >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> o/p of xm li on cvtst2 >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>> [root@cvtst2 ~]# xm li >>>>>>>>>>>>>>>>>>>>>>> Name ID Mem(MiB) >>>>>>>>>>>>>>>>>>>>>>> VCPUs >>>>>>>>>>>>>>>>>>>>>>> State Time(s) >>>>>>>>>>>>>>>>>>>>>>> Domain-0 0 3470 >>>>>>>>>>>>>>>>>>>>>>> 2 >>>>>>>>>>>>>>>>>>>>>>> r----- 31558.9 >>>>>>>>>>>>>>>>>>>>>>> guest1 21 511 >>>>>>>>>>>>>>>>>>>>>>> 1 >>>>>>>>>>>>>>>>>>>>>>> -b---- 7558.2 >>>>>>>>>>>>>>>>>>>>>>> --- >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>>>>>>>>> Paras. >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> On Fri, Sep 25, 2009 at 4:22 PM, brem belguebli >>>>>>>>>>>>>>>>>>>>>>> <brem.belguebli@xxxxxxxxx> wrote: >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> It looks like no. >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> can you send an output of clustat of when the VM is >>>>>>>>>>>>>>>>>>>>>>>> running >>>>>>>>>>>>>>>>>>>>>>>> on >>>>>>>>>>>>>>>>>>>>>>>> multiple nodes at the same time? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> And by the way, another one after having stopped >>>>>>>>>>>>>>>>>>>>>>>> (clusvcadm >>>>>>>>>>>>>>>>>>>>>>>> -s vm:guest1) ? >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> 2009/9/25 Paras pradhan <pradhanparas@xxxxxxxxx>: >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Anyone having issue as mine? Virtual machine >>>>>>>>>>>>>>>>>>>>>>>>> service >>>>>>>>>>>>>>>>>>>>>>>>> is >>>>>>>>>>>>>>>>>>>>>>>>> not >>>>>>>>>>>>>>>>>>>>>>>>> being >>>>>>>>>>>>>>>>>>>>>>>>> properly handled by the cluster. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> Thanks >>>>>>>>>>>>>>>>>>>>>>>>> Paras. >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> On Mon, Sep 21, 2009 at 9:55 AM, Paras pradhan >>>>>>>>>>>>>>>>>>>>>>>>> <pradhanparas@xxxxxxxxx> wrote: >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Ok.. here is my cluster.conf file >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>> [root@cvtst1 cluster]# more cluster.conf >>>>>>>>>>>>>>>>>>>>>>>>>> <?xml version="1.0"?> >>>>>>>>>>>>>>>>>>>>>>>>>> <cluster alias="test" config_version="9" >>>>>>>>>>>>>>>>>>>>>>>>>> name="test"> >>>>>>>>>>>>>>>>>>>>>>>>>> <fence_daemon clean_start="0" >>>>>>>>>>>>>>>>>>>>>>>>>> post_fail_delay="0" >>>>>>>>>>>>>>>>>>>>>>>>>> post_join_delay="3"/> >>>>>>>>>>>>>>>>>>>>>>>>>> <clusternodes> >>>>>>>>>>>>>>>>>>>>>>>>>> <clusternode name="cvtst2" nodeid="1" >>>>>>>>>>>>>>>>>>>>>>>>>> votes="1"> >>>>>>>>>>>>>>>>>>>>>>>>>> <fence/> >>>>>>>>>>>>>>>>>>>>>>>>>> </clusternode> >>>>>>>>>>>>>>>>>>>>>>>>>> <clusternode name="cvtst1" nodeid="2" >>>>>>>>>>>>>>>>>>>>>>>>>> votes="1"> >>>>>>>>>>>>>>>>>>>>>>>>>> <fence/> >>>>>>>>>>>>>>>>>>>>>>>>>> </clusternode> >>>>>>>>>>>>>>>>>>>>>>>>>> <clusternode name="cvtst3" nodeid="3" >>>>>>>>>>>>>>>>>>>>>>>>>> votes="1"> >>>>>>>>>>>>>>>>>>>>>>>>>> <fence/> >>>>>>>>>>>>>>>>>>>>>>>>>> </clusternode> >>>>>>>>>>>>>>>>>>>>>>>>>> </clusternodes> >>>>>>>>>>>>>>>>>>>>>>>>>> <cman/> >>>>>>>>>>>>>>>>>>>>>>>>>> <fencedevices/> >>>>>>>>>>>>>>>>>>>>>>>>>> <rm> >>>>>>>>>>>>>>>>>>>>>>>>>> <failoverdomains> >>>>>>>>>>>>>>>>>>>>>>>>>> <failoverdomain name="myfd1" >>>>>>>>>>>>>>>>>>>>>>>>>> nofailback="0" ordered="1" restricted="0"> >>>>>>>>>>>>>>>>>>>>>>>>>> <failoverdomainnode >>>>>>>>>>>>>>>>>>>>>>>>>> name="cvtst2" priority="3"/> >>>>>>>>>>>>>>>>>>>>>>>>>> <failoverdomainnode >>>>>>>>>>>>>>>>>>>>>>>>>> name="cvtst1" priority="1"/> >>>>>>>>>>>>>>>>>>>>>>>>>> <failoverdomainnode >>>>>>>>>>>>>>>>>>>>>>>>>> name="cvtst3" priority="2"/> >>>>>>>>>>>>>>>>>>>>>>>>>> </failoverdomain> >>>>>>>>>>>>>>>>>>>>>>>>>> </failoverdomains> >>>>>>>>>>>>>>>>>>>>>>>>>> <resources/> >>>>>>>>>>>>>>>>>>>>>>>>>> <vm autostart="1" domain="myfd1" >>>>>>>>>>>>>>>>>>>>>>>>>> exclusive="0" max_restarts="0" >>>>>>>>>>>>>>>>>>>>>>>>>> name="guest1" path="/vms" recovery="r >>>>>>>>>>>>>>>>>>>>>>>>>> estart" restart_expire_time="0"/> >>>>>>>>>>>>>>>>>>>>>>>>>> </rm> >>>>>>>>>>>>>>>>>>>>>>>>>> </cluster> >>>>>>>>>>>>>>>>>>>>>>>>>> [root@cvtst1 cluster]# >>>>>>>>>>>>>>>>>>>>>>>>>> ------ >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> Thanks! >>>>>>>>>>>>>>>>>>>>>>>>>> Paras. >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>> On Sun, Sep 20, 2009 at 9:44 AM, Volker Dormeyer >>>>>>>>>>>>>>>>>>>>>>>>>> <volker@xxxxxxxxxxxx> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> On Fri, Sep 18, 2009 at 05:08:57PM -0500, >>>>>>>>>>>>>>>>>>>>>>>>>>> Paras pradhan <pradhanparas@xxxxxxxxx> wrote: >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>>> I am using cluster suite for HA of xen virtual >>>>>>>>>>>>>>>>>>>>>>>>>>>> machines. >>>>>>>>>>>>>>>>>>>>>>>>>>>> Now I am >>>>>>>>>>>>>>>>>>>>>>>>>>>> having another problem. When I start the my xen >>>>>>>>>>>>>>>>>>>>>>>>>>>> vm >>>>>>>>>>>>>>>>>>>>>>>>>>>> in >>>>>>>>>>>>>>>>>>>>>>>>>>>> one node, it >>>>>>>>>>>>>>>>>>>>>>>>>>>> also starts on other nodes. Which daemon >>>>>>>>>>>>>>>>>>>>>>>>>>>> controls >>>>>>>>>>>>>>>>>>>>>>>>>>>> this? >>>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> This is usually done bei clurgmgrd (which is part >>>>>>>>>>>>>>>>>>>>>>>>>>> of >>>>>>>>>>>>>>>>>>>>>>>>>>> the >>>>>>>>>>>>>>>>>>>>>>>>>>> rgmanager >>>>>>>>>>>>>>>>>>>>>>>>>>> package). To me, this sounds like a configuration >>>>>>>>>>>>>>>>>>>>>>>>>>> problem. Maybe, >>>>>>>>>>>>>>>>>>>>>>>>>>> you can post your cluster.conf? >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> Regards, >>>>>>>>>>>>>>>>>>>>>>>>>>> Volker >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list >>>>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list >>>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list >>>>>>>>>>>>>>>>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list >>>>>>>>>>>>>>>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list >>>>>>>>>>>>>>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list >>>>>>>>>>>>>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>>> Linux-cluster mailing list >>>>>>>>>>>>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>>>> Linux-cluster mailing list >>>>>>>>>>>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>>> Linux-cluster mailing list >>>>>>>>>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>>> Linux-cluster mailing list >>>>>>>>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>>> >>>>>>>>>>>>>>> -- >>>>>>>>>>>>>>> Linux-cluster mailing list >>>>>>>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>>>>>>> >>>>>>>>>>>>>>> >>>>>>>>>>>>>> -- >>>>>>>>>>>>>> Linux-cluster mailing list >>>>>>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>>>>>> >>>>>>>>>>>>>> >>>>>>>>>>>>> -- >>>>>>>>>>>>> Linux-cluster mailing list >>>>>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>>>>> >>>>>>>>>>>>> >>>>>>>>>>>> -- >>>>>>>>>>>> Linux-cluster mailing list >>>>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>>>> >>>>>>>>>>>> >>>>>>>>>>> -- >>>>>>>>>>> Linux-cluster mailing list >>>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>> -- >>>>>>>>>> Linux-cluster mailing list >>>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>>> >>>>>>>>> -- >>>>>>>>> Linux-cluster mailing list >>>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>>> >>>>>>>> -- >>>>>>>> Linux-cluster mailing list >>>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>>> >>>>>>> -- >>>>>>> Linux-cluster mailing list >>>>>>> Linux-cluster@xxxxxxxxxx >>>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>>> >>>>>> -- >>>>>> Linux-cluster mailing list >>>>>> Linux-cluster@xxxxxxxxxx >>>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>>> >>>>> -- >>>>> - Daniela Anzellotti ------------------------------------ >>>>> INFN Roma - tel.: +39.06.49914282 - fax: +39.06.490354 >>>>> e-mail: daniela.anzellotti@xxxxxxxxxxxxx >>>>> --------------------------------------------------------- >>>>> >>>>> -- >>>>> Linux-cluster mailing list >>>>> Linux-cluster@xxxxxxxxxx >>>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>>> >>>> -- >>>> Linux-cluster mailing list >>>> Linux-cluster@xxxxxxxxxx >>>> https://www.redhat.com/mailman/listinfo/linux-cluster >>>> >>> -- >>> - Daniela Anzellotti ------------------------------------ >>> INFN Roma - tel.: +39.06.49914282 - fax: +39.06.490354 >>> e-mail: daniela.anzellotti@xxxxxxxxxxxxx >>> --------------------------------------------------------- >>> >>> -- >>> Linux-cluster mailing list >>> Linux-cluster@xxxxxxxxxx >>> https://www.redhat.com/mailman/listinfo/linux-cluster >>> >> >> -- >> Linux-cluster mailing list >> Linux-cluster@xxxxxxxxxx >> https://www.redhat.com/mailman/listinfo/linux-cluster >> > > -- > - Daniela Anzellotti ------------------------------------ > INFN Roma - tel.: +39.06.49914282 - fax: +39.06.490354 > e-mail: daniela.anzellotti@xxxxxxxxxxxxx > --------------------------------------------------------- > > -- > Linux-cluster mailing list > Linux-cluster@xxxxxxxxxx > https://www.redhat.com/mailman/listinfo/linux-cluster > -- Linux-cluster mailing list Linux-cluster@xxxxxxxxxx https://www.redhat.com/mailman/listinfo/linux-cluster
