Arjun Pandey
2015-01-22 05:39:05 UTC
Any pointers on this would be helpful.
Regards
Arjun
Regards
Arjun
I am running a 2 node cluster(active-standby mode) with this config
CentOS-6.5
pacemaker 1.1.10-14.el6_5.3
pcs 0.9.90-2.el6.centos.3 - (none) noarch
corosync 1.4.1-17.el6_5.1
Online: [ lakshman shatrughan ]
Master/Slave Set: foo-master [foo]
Masters: [ shatrughan ]
Slaves: [ lakshman ]
eth1-CP (ocf::pw:IPaddr): Started shatrughan
eth2-UP (ocf::pw:IPaddr): Started shatrughan
eth3-UPCP (ocf::pw:IPaddr): Started shatrughan
where IPaddr RA is just modified IPAddr2 RA and foo is a multi-state
resource. Additionally i have a
collocation constraint for the IP addr to be collocated with the master.
I have set the migration-threshold as 2 for the VIP. I also have set the
failure-timeout to 15s.
All virtual IP's have following properties
Meta Attrs: failure-timeout=3s migration-threshold=2
Operations: monitor interval=1s (RAN-monitor-interval-1s)
However after some time i notice that the colocation constraint i had
applied has got removed and the resource get's started where foo standby is
located.
From the logs
Dec 17 16:15:32 [27112] lakshman pengine: notice: process_pe_message: Calculated
Transition 3: /var/lib/pacemaker/pengine/pe-input-1297.bz2
do_state_transition: State transition S_POLICY_ENGINE ->
S_TRANSITION_ENGINE [ input=I_PE_SFOOCESS cause=C_IPC_MESSAGE
origin=handle_response ]
Dec 17 16:15:32 [27113] lakshman crmd: info: do_te_invoke: Processing
graph 3 (ref=pe_calc-dc-1418813132-33) derived from
/var/lib/pacemaker/pengine/pe-input-1297.bz2
Dec 17 16:15:32 [27113] lakshman crmd: notice: te_rsc_command: Initiating
action 12: monitor foo_monitor_10000 on lakshman (local)
Dec 17 16:15:32 [27113] lakshman crmd: info: do_lrm_rsc_op: Performing
key=12:3:8:62cbe40f-174d-458b-8e52-8581e57245a0 op=foo_monitor_10000
Dec 17 16:15:32 [27113] lakshman crmd: notice: process_lrm_event: LRM
operation foo_monitor_10000 (call=77, rc=8, cib-update=52, confirmed=false)
master
OK (rc=0, origin=local/crmd/52, version=0.652.73)
Dec 17 16:15:32 [27113] lakshman crmd: info: match_graph_event: Action
foo_monitor_10000 (12) confirmed on lakshman (rc=0)
Dec 17 16:15:32 [27113] lakshman crmd: notice: run_graph: Transition
3 (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-1297.bz2): Complete
Dec 17 16:15:32 [27113] lakshman crmd: info: do_log: FSA: Input
I_TE_SFOOCESS from notify_crmd() received in state S_TRANSITION_ENGINE
do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [
input=I_TE_SFOOCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28098 id=4d1ef70d-3f75-4308-9c1b-cc410a6e6940
cib_process_request: Completed cib_query operation for section 'all': OK
(rc=0, origin=local/crm_resource/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28102 id=14b4a917-4df1-46ed-82e3-0a023b92ed11
cib_process_request: Completed cib_query operation for section
origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28103 id=4ab59dd4-7f13-4ae6-84cf-ae9c2de5e246
cib_process_request: Completed cib_query operation for section
origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28104 id=7d24f5e0-2ae2-47be-a88e-1635e1268d3c
cib_process_request: Completed cib_query operation for section
origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28105 id=c583545c-e590-4f5a-a865-e063d21f35cd
cib_process_request: Completed cib_query operation for section
origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28106 id=d7dc6094-4c48-4246-94b3-dc2b0a12866f
cib_process_request: Completed cib_query operation for section
origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28107 id=757a2e15-a1d4-4f3b-b93a-0051b5a9be15
cib_process_request: Completed cib_query operation for section
origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28108 id=1c9bfb99-bd55-42dc-b1bd-df32d560af90
cib_process_request: Completed cib_query operation for section
origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28109 id=1b7ede71-66ec-43ad-9754-b204f367ccc3
cib_process_request: Completed cib_query operation for section 'all': OK
(rc=0, origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28110 id=ebd97f0f-81fc-4fd3-adae-30fb9784b563
cib_process_request: Completed cib_query operation for section
//constraints: OK (rc=0, origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28111 id=e4891335-7a9b-4189-ba3d-03008b905664
0.652.73 -> 0.653.1 from lakshman
--- 0.652.73
+++ 0.653.1 dfc649135fe2d861910a19c9f983cc4a
Dec 17 16:15:32 [27108] lakshman cib: notice: cib:diff: --
<rsc_colocation id="colocation-Mgmt_Float-foo-master-INFINITY"
rsc="Mgmt_Float" rsc-role="Started" score="INFINITY" with-rsc="foo-master"
with-rsc-role="Master"/>
PFA the cluster-log file
I can't really make out why the constraint got removed in the first place.
Thanks for help in advance.
Regards
Arjun
CentOS-6.5
pacemaker 1.1.10-14.el6_5.3
pcs 0.9.90-2.el6.centos.3 - (none) noarch
corosync 1.4.1-17.el6_5.1
Online: [ lakshman shatrughan ]
Master/Slave Set: foo-master [foo]
Masters: [ shatrughan ]
Slaves: [ lakshman ]
eth1-CP (ocf::pw:IPaddr): Started shatrughan
eth2-UP (ocf::pw:IPaddr): Started shatrughan
eth3-UPCP (ocf::pw:IPaddr): Started shatrughan
where IPaddr RA is just modified IPAddr2 RA and foo is a multi-state
resource. Additionally i have a
collocation constraint for the IP addr to be collocated with the master.
I have set the migration-threshold as 2 for the VIP. I also have set the
failure-timeout to 15s.
All virtual IP's have following properties
Meta Attrs: failure-timeout=3s migration-threshold=2
Operations: monitor interval=1s (RAN-monitor-interval-1s)
However after some time i notice that the colocation constraint i had
applied has got removed and the resource get's started where foo standby is
located.
From the logs
Dec 17 16:15:32 [27112] lakshman pengine: notice: process_pe_message: Calculated
Transition 3: /var/lib/pacemaker/pengine/pe-input-1297.bz2
do_state_transition: State transition S_POLICY_ENGINE ->
S_TRANSITION_ENGINE [ input=I_PE_SFOOCESS cause=C_IPC_MESSAGE
origin=handle_response ]
Dec 17 16:15:32 [27113] lakshman crmd: info: do_te_invoke: Processing
graph 3 (ref=pe_calc-dc-1418813132-33) derived from
/var/lib/pacemaker/pengine/pe-input-1297.bz2
Dec 17 16:15:32 [27113] lakshman crmd: notice: te_rsc_command: Initiating
action 12: monitor foo_monitor_10000 on lakshman (local)
Dec 17 16:15:32 [27113] lakshman crmd: info: do_lrm_rsc_op: Performing
key=12:3:8:62cbe40f-174d-458b-8e52-8581e57245a0 op=foo_monitor_10000
Dec 17 16:15:32 [27113] lakshman crmd: notice: process_lrm_event: LRM
operation foo_monitor_10000 (call=77, rc=8, cib-update=52, confirmed=false)
master
OK (rc=0, origin=local/crmd/52, version=0.652.73)
Dec 17 16:15:32 [27113] lakshman crmd: info: match_graph_event: Action
foo_monitor_10000 (12) confirmed on lakshman (rc=0)
Dec 17 16:15:32 [27113] lakshman crmd: notice: run_graph: Transition
3 (Complete=1, Pending=0, Fired=0, Skipped=0, Incomplete=0,
Source=/var/lib/pacemaker/pengine/pe-input-1297.bz2): Complete
Dec 17 16:15:32 [27113] lakshman crmd: info: do_log: FSA: Input
I_TE_SFOOCESS from notify_crmd() received in state S_TRANSITION_ENGINE
do_state_transition: State transition S_TRANSITION_ENGINE -> S_IDLE [
input=I_TE_SFOOCESS cause=C_FSA_INTERNAL origin=notify_crmd ]
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28098 id=4d1ef70d-3f75-4308-9c1b-cc410a6e6940
cib_process_request: Completed cib_query operation for section 'all': OK
(rc=0, origin=local/crm_resource/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28102 id=14b4a917-4df1-46ed-82e3-0a023b92ed11
cib_process_request: Completed cib_query operation for section
origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28103 id=4ab59dd4-7f13-4ae6-84cf-ae9c2de5e246
cib_process_request: Completed cib_query operation for section
origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28104 id=7d24f5e0-2ae2-47be-a88e-1635e1268d3c
cib_process_request: Completed cib_query operation for section
origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28105 id=c583545c-e590-4f5a-a865-e063d21f35cd
cib_process_request: Completed cib_query operation for section
origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28106 id=d7dc6094-4c48-4246-94b3-dc2b0a12866f
cib_process_request: Completed cib_query operation for section
origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28107 id=757a2e15-a1d4-4f3b-b93a-0051b5a9be15
cib_process_request: Completed cib_query operation for section
origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28108 id=1c9bfb99-bd55-42dc-b1bd-df32d560af90
cib_process_request: Completed cib_query operation for section
origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28109 id=1b7ede71-66ec-43ad-9754-b204f367ccc3
cib_process_request: Completed cib_query operation for section 'all': OK
(rc=0, origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28110 id=ebd97f0f-81fc-4fd3-adae-30fb9784b563
cib_process_request: Completed cib_query operation for section
//constraints: OK (rc=0, origin=local/cibadmin/2, version=0.652.73)
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_destroy: Destroying
0 events
Dec 17 16:15:32 [27108] lakshman cib: info: crm_client_new: Connecting
0x20bb1d0 for uid=0 gid=0 pid=28111 id=e4891335-7a9b-4189-ba3d-03008b905664
0.652.73 -> 0.653.1 from lakshman
--- 0.652.73
+++ 0.653.1 dfc649135fe2d861910a19c9f983cc4a
Dec 17 16:15:32 [27108] lakshman cib: notice: cib:diff: --
<rsc_colocation id="colocation-Mgmt_Float-foo-master-INFINITY"
rsc="Mgmt_Float" rsc-role="Started" score="INFINITY" with-rsc="foo-master"
with-rsc-role="Master"/>
PFA the cluster-log file
I can't really make out why the constraint got removed in the first place.
Thanks for help in advance.
Regards
Arjun