[原]安装及配置红旗高可用服务器 HA 5.0 [3] - 生成HA策略文件
[原]安装及配置红旗高可用服务器 HA 5.0 [5] - 使用WebManager
Dec
12
[原]安装及配置红旗高可用服务器 HA 5.0 [4] - 运行集群 
八、切换测试
切换测试包括几方面的内容：手动切换、测试应用监控、测试网络监控等。
1、手动切换
在运行有应用的服务器上执行手动切换的命令：
[root@hatest1 ~]# clpgrp -m httpd
hatest1的日志：
引用
Dec 12 15:41:59 hatest1 rfhacluster: <type: apisv><event: 36> There was a request to move group(httpd) from the clpgrp command.
Dec 12 15:41:59 hatest1 rfhacluster: <type: rc><event: 20> The stop processing of a group httpd started.
Dec 12 15:41:59 hatest1 rfhacluster: <type: rc><event: 40> The stop processing of a resource monexec started.
Dec 12 15:42:00 hatest1 rfhacluster: <type: rm><event: 2> Monitor pidw stop.
Dec 12 15:42:01 hatest1 rfhacluster: <type: httpmon><event: 52> Trial Period of clp_httpmon is till 12/31/2008 (mm/dd/yyyy).
Dec 12 15:42:02 hatest1 rfhacluster: <type: rc><event: 41> The stop processing of a resource monexec ended.
Dec 12 15:42:02 hatest1 rfhacluster: <type: rc><event: 40> The stop processing of a resource exec started.
Dec 12 15:42:03 hatest1 rfhacluster: <type: httpmon><event: 3> The clp_httpmon will stop watching HTTP daemon. [ID:httpwatch]
Dec 12 15:42:04 hatest1 httpd: httpd shutdown succeeded
Dec 12 15:41:59 hatest1 rfhacluster: <type: rc><event: 20> The stop processing of a group httpd started.
Dec 12 15:41:59 hatest1 rfhacluster: <type: rc><event: 40> The stop processing of a resource monexec started.
Dec 12 15:42:00 hatest1 rfhacluster: <type: rm><event: 2> Monitor pidw stop.
Dec 12 15:42:01 hatest1 rfhacluster: <type: httpmon><event: 52> Trial Period of clp_httpmon is till 12/31/2008 (mm/dd/yyyy).
Dec 12 15:42:02 hatest1 rfhacluster: <type: rc><event: 41> The stop processing of a resource monexec ended.
Dec 12 15:42:02 hatest1 rfhacluster: <type: rc><event: 40> The stop processing of a resource exec started.
Dec 12 15:42:03 hatest1 rfhacluster: <type: httpmon><event: 3> The clp_httpmon will stop watching HTTP daemon. [ID:httpwatch]
Dec 12 15:42:04 hatest1 httpd: httpd shutdown succeeded
hatest2的日志：
引用
Dec 12 15:41:59 hatest2 rfhacluster: <type: rc><event: 50> The move processing of a group httpd started.
Dec 12 15:42:11 hatest2 rfhacluster: <type: rc><event: 10> The start processing of a group httpd started.
Dec 12 15:42:11 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource fip started.
Dec 12 15:42:14 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource fip ended.
Dec 12 15:42:14 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource disk started.
Dec 12 15:42:14 hatest2 kernel: kjournald starting. Commit interval 5 seconds
Dec 12 15:42:14 hatest2 kernel: EXT3 FS on sdb1, internal journal
Dec 12 15:42:14 hatest2 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Dec 12 15:42:15 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource disk ended.
Dec 12 15:42:15 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource exec started.
Dec 12 15:42:16 hatest2 sshd(pam_unix)[8218]: session opened for user root by root(uid=0)
Dec 12 15:42:16 hatest2 httpd: httpd startup succeeded
Dec 12 15:42:17 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource exec ended.
Dec 12 15:42:17 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource monexec started.
Dec 12 15:42:18 hatest2 rfhacluster: <type: httpmon><event: 52> Trial Period of clp_httpmon is till 12/31/2008 (mm/dd/yyyy).
Dec 12 15:42:19 hatest2 rfhacluster: <type: rm><event: 1> Monitor pidw start.
Dec 12 15:42:20 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource monexec ended.
Dec 12 15:42:20 hatest2 rfhacluster: <type: rc><event: 11> The start processing of a group httpd ended.
Dec 12 15:42:20 hatest2 rfhacluster: <type: rc><event: 51> The move processing of a group httpd ended.
Dec 12 15:42:20 hatest2 rfhacluster: <type: httpmon><event: 1> The clp_httpmon has started watching HTTP. [ID:httpwatch]
Dec 12 15:42:21 hatest2 rfhacluster: <type: httpmon><event: 2> The clp_httpmon is going to watch HTTP daemon. [ID:httpwatch]
Dec 12 15:42:11 hatest2 rfhacluster: <type: rc><event: 10> The start processing of a group httpd started.
Dec 12 15:42:11 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource fip started.
Dec 12 15:42:14 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource fip ended.
Dec 12 15:42:14 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource disk started.
Dec 12 15:42:14 hatest2 kernel: kjournald starting. Commit interval 5 seconds
Dec 12 15:42:14 hatest2 kernel: EXT3 FS on sdb1, internal journal
Dec 12 15:42:14 hatest2 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Dec 12 15:42:15 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource disk ended.
Dec 12 15:42:15 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource exec started.
Dec 12 15:42:16 hatest2 sshd(pam_unix)[8218]: session opened for user root by root(uid=0)
Dec 12 15:42:16 hatest2 httpd: httpd startup succeeded
Dec 12 15:42:17 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource exec ended.
Dec 12 15:42:17 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource monexec started.
Dec 12 15:42:18 hatest2 rfhacluster: <type: httpmon><event: 52> Trial Period of clp_httpmon is till 12/31/2008 (mm/dd/yyyy).
Dec 12 15:42:19 hatest2 rfhacluster: <type: rm><event: 1> Monitor pidw start.
Dec 12 15:42:20 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource monexec ended.
Dec 12 15:42:20 hatest2 rfhacluster: <type: rc><event: 11> The start processing of a group httpd ended.
Dec 12 15:42:20 hatest2 rfhacluster: <type: rc><event: 51> The move processing of a group httpd ended.
Dec 12 15:42:20 hatest2 rfhacluster: <type: httpmon><event: 1> The clp_httpmon has started watching HTTP. [ID:httpwatch]
Dec 12 15:42:21 hatest2 rfhacluster: <type: httpmon><event: 2> The clp_httpmon is going to watch HTTP daemon. [ID:httpwatch]
切换成功：
引用
[root@hatest2 ~]# clpstat
======================== CLUSTER STATUS ===========================
Cluster : rfgz
<server>
hatest1 .........: Online
lanhb1 : Normal LAN Heartbeat
lanhb2 : Normal LAN Heartbeat
*hatest2 .........: Online
lanhb1 : Normal LAN Heartbeat
lanhb2 : Normal LAN Heartbeat
<group>
httpd ...........: Online httpd Group
current : hatest2
disk : Online
exec : Online
fip : Online
monexec : Online
<monitor>
pidw : Normal
userw : Normal user mode monitor
=====================================================================
======================== CLUSTER STATUS ===========================
Cluster : rfgz
<server>
hatest1 .........: Online
lanhb1 : Normal LAN Heartbeat
lanhb2 : Normal LAN Heartbeat
*hatest2 .........: Online
lanhb1 : Normal LAN Heartbeat
lanhb2 : Normal LAN Heartbeat
<group>
httpd ...........: Online httpd Group
current : hatest2
disk : Online
exec : Online
fip : Online
monexec : Online
<monitor>
pidw : Normal
userw : Normal user mode monitor
=====================================================================
再次查看应用情况：
证明，应用已经运行在hatest2上了。
2、测试应用监控
可采用强制终止应用程序的方式，查看HA是否会重新激活应用。在运行有应用的机器上运行：
引用
[root@hatest2 ~]# service httpd stop
停止 httpd： [ 确定 ]
停止 httpd： [ 确定 ]
查看日志：
引用
Dec 12 15:49:33 hatest2 rfhacluster: <type: httpmon><event: 72> The clp_httpmon has detected network error(connect error[111]). [ID:httpwatch]
Dec 12 15:49:34 hatest2 rfhacluster: <type: httpmon><event: 6> The clp_httpmon will terminate. [ID:httpwatch]
Dec 12 15:49:40 hatest2 rfhacluster: <type: rm><event: 9> Monitor pidw failed. (1 : Process does not exist. (pid=8263))
Dec 12 15:49:40 hatest2 rfhacluster: <type: apisv><event: 34> There was a request to restart group(httpd) from the clprm process.
Dec 12 15:49:40 hatest2 rfhacluster: <type: rc><event: 70> The restart processing of a group httpd started.
Dec 12 15:49:40 hatest2 rfhacluster: <type: rc><event: 20> The stop processing of a group httpd started.
Dec 12 15:49:40 hatest2 rfhacluster: <type: rc><event: 40> The stop processing of a resource monexec started.
Dec 12 15:49:40 hatest2 rfhacluster: <type: rm><event: 2> Monitor pidw stop.
Dec 12 15:49:42 hatest2 rfhacluster: <type: httpmon><event: 52> Trial Period of clp_httpmon is till 12/31/2008 (mm/dd/yyyy).
Dec 12 15:49:42 hatest2 rfhacluster: <type: rc><event: 41> The stop processing of a resource monexec ended.
Dec 12 15:49:42 hatest2 rfhacluster: <type: rc><event: 40> The stop processing of a resource exec started.
Dec 12 15:49:43 hatest2 httpd: httpd shutdown failed
Dec 12 15:49:45 hatest2 rfhacluster: <type: rc><event: 41> The stop processing of a resource exec ended.
Dec 12 15:49:45 hatest2 rfhacluster: <type: rc><event: 40> The stop processing of a resource disk started.
Dec 12 15:49:46 hatest2 rfhacluster: <type: rc><event: 41> The stop processing of a resource disk ended.
Dec 12 15:49:46 hatest2 rfhacluster: <type: rc><event: 40> The stop processing of a resource fip started.
Dec 12 15:49:50 hatest2 rfhacluster: <type: rc><event: 41> The stop processing of a resource fip ended.
Dec 12 15:49:50 hatest2 rfhacluster: <type: rc><event: 21> The stop processing of a group httpd ended.
Dec 12 15:49:50 hatest2 rfhacluster: <type: rc><event: 10> The start processing of a group httpd started.
Dec 12 15:49:50 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource fip started.
Dec 12 15:49:53 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource fip ended.
Dec 12 15:49:53 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource disk started.
Dec 12 15:49:53 hatest2 kernel: kjournald starting. Commit interval 5 seconds
Dec 12 15:49:53 hatest2 kernel: EXT3 FS on sdb1, internal journal
Dec 12 15:49:53 hatest2 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Dec 12 15:49:54 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource disk ended.
Dec 12 15:49:54 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource exec started.
Dec 12 15:49:55 hatest2 httpd: httpd startup succeeded
Dec 12 15:49:56 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource exec ended.
Dec 12 15:49:56 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource monexec started.
Dec 12 15:49:57 hatest2 rfhacluster: <type: httpmon><event: 52> Trial Period of clp_httpmon is till 12/31/2008 (mm/dd/yyyy).
Dec 12 15:49:58 hatest2 rfhacluster: <type: rm><event: 1> Monitor pidw start.
Dec 12 15:49:58 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource monexec ended.
Dec 12 15:49:58 hatest2 rfhacluster: <type: rc><event: 11> The start processing of a group httpd ended.
Dec 12 15:49:59 hatest2 rfhacluster: <type: rc><event: 71> The restart processing of a group httpd ended.
Dec 12 15:49:59 hatest2 rfhacluster: <type: rm><event: 13> httpd was restarted for failure in pidw.
Dec 12 15:49:59 hatest2 rfhacluster: <type: httpmon><event: 1> The clp_httpmon has started watching HTTP. [ID:httpwatch]
Dec 12 15:50:00 hatest2 rfhacluster: <type: httpmon><event: 2> The clp_httpmon is going to watch HTTP daemon. [ID:httpwatch]
Dec 12 15:49:34 hatest2 rfhacluster: <type: httpmon><event: 6> The clp_httpmon will terminate. [ID:httpwatch]
Dec 12 15:49:40 hatest2 rfhacluster: <type: rm><event: 9> Monitor pidw failed. (1 : Process does not exist. (pid=8263))
Dec 12 15:49:40 hatest2 rfhacluster: <type: apisv><event: 34> There was a request to restart group(httpd) from the clprm process.
Dec 12 15:49:40 hatest2 rfhacluster: <type: rc><event: 70> The restart processing of a group httpd started.
Dec 12 15:49:40 hatest2 rfhacluster: <type: rc><event: 20> The stop processing of a group httpd started.
Dec 12 15:49:40 hatest2 rfhacluster: <type: rc><event: 40> The stop processing of a resource monexec started.
Dec 12 15:49:40 hatest2 rfhacluster: <type: rm><event: 2> Monitor pidw stop.
Dec 12 15:49:42 hatest2 rfhacluster: <type: httpmon><event: 52> Trial Period of clp_httpmon is till 12/31/2008 (mm/dd/yyyy).
Dec 12 15:49:42 hatest2 rfhacluster: <type: rc><event: 41> The stop processing of a resource monexec ended.
Dec 12 15:49:42 hatest2 rfhacluster: <type: rc><event: 40> The stop processing of a resource exec started.
Dec 12 15:49:43 hatest2 httpd: httpd shutdown failed
Dec 12 15:49:45 hatest2 rfhacluster: <type: rc><event: 41> The stop processing of a resource exec ended.
Dec 12 15:49:45 hatest2 rfhacluster: <type: rc><event: 40> The stop processing of a resource disk started.
Dec 12 15:49:46 hatest2 rfhacluster: <type: rc><event: 41> The stop processing of a resource disk ended.
Dec 12 15:49:46 hatest2 rfhacluster: <type: rc><event: 40> The stop processing of a resource fip started.
Dec 12 15:49:50 hatest2 rfhacluster: <type: rc><event: 41> The stop processing of a resource fip ended.
Dec 12 15:49:50 hatest2 rfhacluster: <type: rc><event: 21> The stop processing of a group httpd ended.
Dec 12 15:49:50 hatest2 rfhacluster: <type: rc><event: 10> The start processing of a group httpd started.
Dec 12 15:49:50 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource fip started.
Dec 12 15:49:53 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource fip ended.
Dec 12 15:49:53 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource disk started.
Dec 12 15:49:53 hatest2 kernel: kjournald starting. Commit interval 5 seconds
Dec 12 15:49:53 hatest2 kernel: EXT3 FS on sdb1, internal journal
Dec 12 15:49:53 hatest2 kernel: EXT3-fs: mounted filesystem with ordered data mode.
Dec 12 15:49:54 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource disk ended.
Dec 12 15:49:54 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource exec started.
Dec 12 15:49:55 hatest2 httpd: httpd startup succeeded
Dec 12 15:49:56 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource exec ended.
Dec 12 15:49:56 hatest2 rfhacluster: <type: rc><event: 30> The start processing of a resource monexec started.
Dec 12 15:49:57 hatest2 rfhacluster: <type: httpmon><event: 52> Trial Period of clp_httpmon is till 12/31/2008 (mm/dd/yyyy).
Dec 12 15:49:58 hatest2 rfhacluster: <type: rm><event: 1> Monitor pidw start.
Dec 12 15:49:58 hatest2 rfhacluster: <type: rc><event: 31> The start processing of a resource monexec ended.
Dec 12 15:49:58 hatest2 rfhacluster: <type: rc><event: 11> The start processing of a group httpd ended.
Dec 12 15:49:59 hatest2 rfhacluster: <type: rc><event: 71> The restart processing of a group httpd ended.
Dec 12 15:49:59 hatest2 rfhacluster: <type: rm><event: 13> httpd was restarted for failure in pidw.
Dec 12 15:49:59 hatest2 rfhacluster: <type: httpmon><event: 1> The clp_httpmon has started watching HTTP. [ID:httpwatch]
Dec 12 15:50:00 hatest2 rfhacluster: <type: httpmon><event: 2> The clp_httpmon is going to watch HTTP daemon. [ID:httpwatch]
可见，HA把出现问题的应用重新启动了。
3、测试网卡监控
由于VMware虚拟机提供的pcnet32网卡不支持mii，所以，我的配置文件中，没有加入对网卡的监控。否则，会看到下面信息：
引用
Dec 12 15:23:46 hatest1 rfhacluster: <type: rm><event: 9> Monitor miiw failed. (13 : MII was not supported or no such device. Device=eth0)Dec 12 15:23:46 hatest1 rfhacluster: <type: apisv><event: 38> There was a request to failover group(httpd) from the clprm process.
Dec 12 15:23:46 hatest1 rfhacluster: <type: rc><event: 20> The stop processing of a group httpd started.
Dec 12 15:23:46 hatest1 rfhacluster: <type: rc><event: 20> The stop processing of a group httpd started.
但如果你是在真实服务器上配置的话，可在加入网卡监控后，拔取服务器的对外网卡（根据配置中内容决定），HA就会报miiw failed错误，并发生切换了。
同样的，如果在HA中加入网关IP的监控等，在切换时也会产生类似的提示信息。
4、测试关闭机器
把运行有应用的服务器关闭：
[root@hatest2 ~]# shutdown -h 0
hatest2的日志显示：
引用
Dec 12 16:00:04 hatest2 rfhacluster: <type: apisv><event: 11> There was a request to shutdown server from the clpdown command.
Dec 12 16:00:04 hatest2 rfhacluster: <type: rc><event: 91> Shutdown the server.
Dec 12 16:00:04 hatest2 rfhacluster: <type: rc><event: 20> The stop processing of a group httpd started.
......
Dec 12 16:00:16 hatest2 rfhacluster: <type: pm><event: 30> There was a request to halt system from the internal.
Dec 12 16:00:19 hatest2 rfhacluster: <type: rm><event: 2> Monitor userw stop.
Dec 12 16:00:21 hatest2 rfhacluster: <type: pm><event: 3> Shutdown stall monitor has started...
Dec 12 16:00:23 hatest2 shutdown: shutting down for system halt
Dec 12 16:00:24 hatest2 rfhacluster: <type: pm><event: 2> Cluster daemon is shutting down...
Dec 12 16:00:04 hatest2 rfhacluster: <type: rc><event: 91> Shutdown the server.
Dec 12 16:00:04 hatest2 rfhacluster: <type: rc><event: 20> The stop processing of a group httpd started.
......
Dec 12 16:00:16 hatest2 rfhacluster: <type: pm><event: 30> There was a request to halt system from the internal.
Dec 12 16:00:19 hatest2 rfhacluster: <type: rm><event: 2> Monitor userw stop.
Dec 12 16:00:21 hatest2 rfhacluster: <type: pm><event: 3> Shutdown stall monitor has started...
Dec 12 16:00:23 hatest2 shutdown: shutting down for system halt
Dec 12 16:00:24 hatest2 rfhacluster: <type: pm><event: 2> Cluster daemon is shutting down...
再看看另一台hatest1的日志：
引用
Dec 12 16:01:52 hatest1 rfhacluster: <type: nm><event: 4> Resource lanhb1 of server hatest2 down.
Dec 12 16:01:52 hatest1 rfhacluster: <type: nm><event: 4> Resource lanhb2 of server hatest2 down.
Dec 12 16:01:52 hatest1 rfhacluster: <type: nm><event: 2> Server hatest2 down.
Dec 12 16:01:53 hatest1 rfhacluster: <type: rc><event: 60> The failover processing of a group httpd started.
Dec 12 16:01:53 hatest1 rfhacluster: <type: rc><event: 10> The start processing of a group httpd started.
......
Dec 12 16:02:03 hatest1 rfhacluster: <type: rc><event: 11> The start processing of a group httpd ended.
Dec 12 16:02:03 hatest1 rfhacluster: <type: rc><event: 61> The failover processing of a group httpd ended.
Dec 12 16:02:03 hatest1 rfhacluster: <type: httpmon><event: 1> The clp_httpmon has started watching HTTP. [ID:httpwatch]
Dec 12 16:02:04 hatest1 rfhacluster: <type: httpmon><event: 2> The clp_httpmon is going to watch HTTP daemon. [ID:httpwatch]
Dec 12 16:01:52 hatest1 rfhacluster: <type: nm><event: 4> Resource lanhb2 of server hatest2 down.
Dec 12 16:01:52 hatest1 rfhacluster: <type: nm><event: 2> Server hatest2 down.
Dec 12 16:01:53 hatest1 rfhacluster: <type: rc><event: 60> The failover processing of a group httpd started.
Dec 12 16:01:53 hatest1 rfhacluster: <type: rc><event: 10> The start processing of a group httpd started.
......
Dec 12 16:02:03 hatest1 rfhacluster: <type: rc><event: 11> The start processing of a group httpd ended.
Dec 12 16:02:03 hatest1 rfhacluster: <type: rc><event: 61> The failover processing of a group httpd ended.
Dec 12 16:02:03 hatest1 rfhacluster: <type: httpmon><event: 1> The clp_httpmon has started watching HTTP. [ID:httpwatch]
Dec 12 16:02:04 hatest1 rfhacluster: <type: httpmon><event: 2> The clp_httpmon is going to watch HTTP daemon. [ID:httpwatch]
HA状态：
引用
[root@hatest1 ~]# clpstat
======================== CLUSTER STATUS ===========================
Cluster : rfgz
<server>
*hatest1 .........: Caution
lanhb1 : Caution LAN Heartbeat
lanhb2 : Caution LAN Heartbeat
hatest2 .........: Offline
lanhb1 : Unknown LAN Heartbeat
lanhb2 : Unknown LAN Heartbeat
<group>
httpd ...........: Online httpd Group
current : hatest1
disk : Online
exec : Online
fip : Online
monexec : Online
<monitor>
pidw : Normal
userw : Normal user mode monitor
=====================================================================
======================== CLUSTER STATUS ===========================
Cluster : rfgz
<server>
*hatest1 .........: Caution
lanhb1 : Caution LAN Heartbeat
lanhb2 : Caution LAN Heartbeat
hatest2 .........: Offline
lanhb1 : Unknown LAN Heartbeat
lanhb2 : Unknown LAN Heartbeat
<group>
httpd ...........: Online httpd Group
current : hatest1
disk : Online
exec : Online
fip : Online
monexec : Online
<monitor>
pidw : Normal
userw : Normal user mode monitor
=====================================================================
应用切换到正常的服务器上了：
至此，基本可验证HA集群运行状态正常。
内文分页： 
