In the beginning
An important LAMP project had been hosted on a wonderfully stable, active/passive cluster with shared-nothing storage. The infrastructure consisted of CentOS 5.x for an OS, DRBD 8.2 for replicated storage, Heartbeat 2 with CRM for cluster messaging and resource management. Resource fencing was provided by DOPD and node fencing via STONITH (IPMI.) Dell Poweredge R610 servers provided the metal.
Putting the cluster together was a relatively painless exercise, all it took was a smart intern (shout out to Val Komarov, a senior CS major at Virginia Tech) and some time to configure it. While somewhat limited, the documentation and guides available were enough to get us going with confidence and no kernel panics.
Time passed and inevitably the LAMP site was rewritten, prompting a move to CentOS 6.x for developer support. I wanted to retain all of the cluster functionality, reinventing as little as possible. How much could possibly change?
Heartbeat, what happened?!
Previously we had made use of Heartbeat 2 for clustering functionality. The Linux-HA project page provided an easy to use implementation guide, and relatively straight forward and unified approach for understanding how the cluster functioned. Not surprisingly, revisiting the site came to reveal that most of the functionality had been decoupled and forked into several sub parts, namely Pacemaker, Corosync, Cluster Glue and Resource Agents. At this point your head might be spinning!
So how do I create a cluster now? I found most of the relevant information for the new implementation at the Pacemaker site. Unfortunately, the number of versions, shells and tools listed on the site made it difficult to choose the ‘best’ path for building the new cluster. Previously you had little choice but to just install the Heartbeat packages, and possibly opt to use CRM. Furthermore, it is now recommended (by Pacemaker) to use Corosync instead of Heartbeat for cluster messaging. Confused yet?
DRBD – 8.3, or 8.4?
I always like to choose stability over features when it comes to disk and data subsystems. Initially I opted to deploy DRBD 8.3 from a 3rd party repository. Unfortunately, I found myself removing 8.3 and running 8.4 to quickly resolve kernel panics when implementing resource level fencing scripts. Unfortunately this also introduced some minor changes to the DRBD configuration file and DRBDADM tool.
CRMSH or PCS, which shell?
Pacemaker supports multiple command line interfaces for creating and modifying cluster resources. While PCS seems to be very active in development, I opted for CRMSH being that there is more reference material for it and it seems to be a logical progression of Heartbeat 2’s CRM commands.
yum install make autoconf libxslt libibverbs librdmacm pkgconfig libtool intltool gettext-devel glib2-devel python-devel libxml2-devel pam-devel ncurses-devel pygtk2 libtool-ltdl libqb libtool-ltdl-devel gnutls-devel swig cluster-glue-libs-devel pacemaker-libs-devel
Edit corosync files: