I am testing FSS (Flexible Shared Storage) on SF 6.1 on RH 5.5 in a Virtual Box VM and when I try to start CVM on the remote node I get:
VCS ERROR V-16-20006-1005 (r55v61b) CVMCluster:cvm_clus:monitor:node - state: out of cluster reason: Disk for disk group not found: retry to add a node failed
Here is my setup:
Node A is master with a local (sdd) and remote disk (B_sdd)
[root@r55v61a ~]# vxdctl -c mode mode: enabled: cluster active - MASTER master: r55v61a [root@r55v61a ~]# vxdisk list DEVICE TYPE DISK GROUP STATUS B_sdd auto:cdsdisk - - online remote sda auto:none - - online invalid sdb auto:none - - online invalid sdc auto:cdsdisk - - online sdd auto:cdsdisk - - online exported
Node B is the slave, and sees local (sdd) and remote disk (A_sdd)
[root@r55v61b ~]# vxdisk list DEVICE TYPE DISK GROUP STATUS A_sdd auto:cdsdisk - - online remote sda auto:none - - online invalid sdb auto:none - - online invalid sdc auto:cdsdisk - - online sdd auto:cdsdisk - - online exported
On node A, I add an FSS diskgroup, so on node A the disk is local
[root@r55v61a ~]# vxdg -s -o fss=on init fss-dg fd1_La=sdd [root@r55v61a ~]# vxdisk list DEVICE TYPE DISK GROUP STATUS B_sdd auto:cdsdisk - - online remote sda auto:none - - online invalid sdb auto:none - - online invalid sdc auto:cdsdisk - - online sdd auto:cdsdisk fd1_La fss-dg online exported shared
And on node B the disk in fss-dg is remote
[root@r55v61b ~]# vxdisk list DEVICE TYPE DISK GROUP STATUS A_sdd auto:cdsdisk fd1_La fss-dg online shared remote sda auto:none - - online invalid sdb auto:none - - online invalid sdc auto:cdsdisk - - online sdd auto:cdsdisk - - online exported
I then stop and start VCS on node B which is when I see the issue:
2014/05/13 12:05:23 VCS INFO V-16-2-13716 (r55v61b) Resource(cvm_clus): Output of the completed operation (online) ============================================== ERROR: ============================================== 2014/05/13 12:05:24 VCS ERROR V-16-20006-1005 (r55v61b) CVMCluster:cvm_clus:monitor:node - state: out of cluster reason: Disk for disk group not found: retry to add a node failed
If I destroy fss-dg diskgroup on node A, then CVM will start on node B, so issue is the FSS diskgroup where it seems CVM cannot find the remote disk in the diskgroup
I can also get round issue by stopping VCS on node A and then CVM will start on node B:
[root@r55v61b ~]# hagrp -online cvm -sys r55v61b [root@r55v61b ~]# vxdisk -o alldgs list DEVICE TYPE DISK GROUP STATUS sda auto:none - - online invalid sdb auto:none - - online invalid sdc auto:cdsdisk - - online sdd auto:cdsdisk - - online exported
If I then start VCS on node A, then B is able to see the FSS diskgroup:
[root@r55v61b ~]# vxdisk list DEVICE TYPE DISK GROUP STATUS A_sdd auto:cdsdisk fd1_La fss-dg online shared remote sda auto:none - - online invalid sdb auto:none - - online invalid sdc auto:cdsdisk - - online sdd auto:cdsdisk - - online exported
I can stop and start VCS on each node when disks are just exported and VCS is able to see disk from other node, but when I create the FSS diskgroup, CVM won't start on the system that has the remote disk - does anybody have any ideas as to why?
Mike