[Pacemaker] Pacemaker 10-15% CPU.

[Karthik]

Hello,
We are using Pacemaker to manage the services that run on a node, as part
of a service management framework, and manage the nodes running the services
as a cluster. One service will be running as 1+1 and other services with be
N+1.

During our testing, we see that the pacemaker processes are taking about
10-15% of the CPU. We would like to know if this is normal and could the
CPU utilization be minimised.

Sample Output of most used CPU process in a Active Manager is

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
189 15766 30.4 0.0 94616 12300 ? Ss 18:01 48:15
/usr/libexec/pacemaker/cib
189 15770 28.9 0.0 118320 20276 ? Ss 18:01 45:53
/usr/libexec/pacemaker/pengine
root 15768 2.6 0.0 76196 3420 ? Ss 18:01 4:12
/usr/libexec/pacemaker/lrmd
root 15767 15.5 0.0 95380 5764 ? Ss 18:01 24:33
/usr/libexec/pacemaker/stonithd

USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
189 15766 30.5 0.0 94616 12300 ? Ss 18:01 49:58
/usr/libexec/pacemaker/cib
189 15770 29.0 0.0 122484 20724 ? Rs 18:01 47:29
/usr/libexec/pacemaker/pengine
root 15768 2.6 0.0 76196 3420 ? Ss 18:01 4:21
/usr/libexec/pacemaker/lrmd
root 15767 15.5 0.0 95380 5764 ? Ss 18:01 25:25
/usr/libexec/pacemaker/stonithd


We also observed that the processes are not distributed equally to all the
available cores and saw that Redhat acknowledging that rhel doesn't
distribute to the available cores efficiently. We are trying to use
IRQbalance to spread the processes to the available cores equally.

Please let us know if there is any way we could minimise the CPU
utilisation. We dont require stonith feature, but there is no way stop that
daemon from running to our knowledge. If that is also possible, please let
us know.

Thanks,
Karthik.


_______________________________________________
Pacemaker mailing list: ***@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

[Vinod Prabhu]

can u look at the files under /var/lib/pacemaker/pengine
these files are generated at periodic intervals.. From these u can know the
reason.
Are the CPU levels continuously high?

Post by Karthik
Hello,
We are using Pacemaker to manage the services that run on a node, as part
of a service management framework, and manage the nodes running the services
as a cluster. One service will be running as 1+1 and other services with be
N+1.
During our testing, we see that the pacemaker processes are taking about
10-15% of the CPU. We would like to know if this is normal and could the
CPU utilization be minimised.
Sample Output of most used CPU process in a Active Manager is
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
189 15766 30.4 0.0 94616 12300 ? Ss 18:01 48:15
/usr/libexec/pacemaker/cib
189 15770 28.9 0.0 118320 20276 ? Ss 18:01 45:53
/usr/libexec/pacemaker/pengine
root 15768 2.6 0.0 76196 3420 ? Ss 18:01 4:12
/usr/libexec/pacemaker/lrmd
root 15767 15.5 0.0 95380 5764 ? Ss 18:01 24:33
/usr/libexec/pacemaker/stonithd
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
189 15766 30.5 0.0 94616 12300 ? Ss 18:01 49:58
/usr/libexec/pacemaker/cib
189 15770 29.0 0.0 122484 20724 ? Rs 18:01 47:29
/usr/libexec/pacemaker/pengine
root 15768 2.6 0.0 76196 3420 ? Ss 18:01 4:21
/usr/libexec/pacemaker/lrmd
root 15767 15.5 0.0 95380 5764 ? Ss 18:01 25:25
/usr/libexec/pacemaker/stonithd
We also observed that the processes are not distributed equally to all the
available cores and saw that Redhat acknowledging that rhel doesn't
distribute to the available cores efficiently. We are trying to use
IRQbalance to spread the processes to the available cores equally.
Please let us know if there is any way we could minimise the CPU
utilisation. We dont require stonith feature, but there is no way stop that
daemon from running to our knowledge. If that is also possible, please let
us know.
Thanks,
Karthik.
_______________________________________________
http://oss.clusterlabs.org/mailman/listinfo/pacemaker
Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

[Karthik]

Yes, Vinod. The CPU levels are continuously high.

In the /var/lib/pacemaker/pengine directory, there are 72 pe-input-xxxx.bz2
files put in every minute. When I decompressed one of the files, I see the
configuration. I am not sure what exact information I have to check for
CPU.

Please let me know if you like to see if some specific information in that
configuration or a whole decompressed file.

Thanks,
Karthik.






_______________________________________________
Pacemaker mailing list: ***@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

[Dejan Muhamedagic]

Hi,

Post by Karthik
Yes, Vinod. The CPU levels are continuously high.
In the /var/lib/pacemaker/pengine directory, there are 72 pe-input-xxxx.bz2
files put in every minute. When I decompressed one of the files, I see the
configuration. I am not sure what exact information I have to check for
CPU.
Please let me know if you like to see if some specific information in that
configuration or a whole decompressed file.

You can use crm_diff to see what's changing between two PE files.
Looks like something's continuously beeing update.

Thanks,

Dejan

Post by Karthik
Thanks,
Karthik.
_______________________________________________
http://oss.clusterlabs.org/mailman/listinfo/pacemaker
Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org

_______________________________________________
Pacemaker mailing list: ***@oss.clusterlabs.org
http://oss.clusterlabs.org/mailman/listinfo/pacemaker

Project Home: http://www.clusterlabs.org
Getting started: http://www.clusterlabs.org/doc/Cluster_from_Scratch.pdf
Bugs: http://bugs.clusterlabs.org