Important info in regards to CentOS 6.5
Due to consequence of multilib support and PATH handling heartbeat’s init scripts wont work on Centos 6.5. The easiest way to fix it is to install heartbeat 3.0.4-2 that is not currently in the epel repo, but can be downloaded from here:
http://koji.fedoraproject.org/koji/taskinfo?taskID=6241294
Note: you will need both heartbeat-3.0.4-2.el6.x86_64.rpm and heartbeat-libs-3.0.4-2.el6.x86_64.rpm packages. In regards to further development of this but you can check this bugzilla bug.
Prerequisites
- Two ‘base’ CentOS servers (architecture is irrelevant)
- A lot of time
Notes about this guide
- I will presume that both CentOS servers were installed following this guide, with a partitioning scheme similar to this.
- Unless otherwise specified, with red for primary and blue for secondary, run each command on both servers.
Before we begin make sure that both servers are up to date:
yum -y update
Installing packages
We need to add a repo first, import the public key:
rpm --import http://elrepo.org/RPM-GPG-KEY-elrepo.org
and than add the repo:
rpm -Uvh http://elrepo.org/elrepo-release-6-5.el6.elrepo.noarch.rpm
now we install drbd, it’s kernel module and heartbeat, by running:
yum -y install drbd84-utils kmod-drbd84 heartbeat
Note: As of writing this document the latest drbd version is 8.4, your milage might differ, change the above command accordingly.
Configuring DRBD
DRBD refers to block devices designed as a building block to form high availability (HA) clusters. This is done by mirroring a whole block device via an assigned network. DRBD can be understood as network based raid-1.
Presuming, that you followed the custom partitioning scheme how-to you should have some free space in your LVM that you can allocate to be used by drbd. Let us create a logical volume, named lv_drbd by running:
lvcreate -L 35GB --name lv_drbd vg_linuxserver
where 35GB is the size of the block device, and vg_linuxserver is your volume group name; usually it’s the same as your host name preceded by a vg_ prefix. You can check your host name by typing:
uname -n
or just check the partitioning table by typing:
fdisk -l
We need to create a config file, by running:
nano /etc/drbd.conf
delete everything inside it (ctrl+k deletes entire lines) and paste this:
global { usage-count no; }
resource repdata {
protocol C;
startup { wfc-timeout 0; degr-wfc-timeout 120; }
disk { on-io-error detach; } # or panic, ...
net { cram-hmac-alg "sha1"; shared-secret "somepassword"; } # don't forget to choose a secret for auth !
syncer { rate 10M; }
on linuxserver1 {
device /dev/drbd0;
disk /dev/vg_linuxserver1/lv_drbd;
address 192.168.10.2:7788;
meta-disk internal;
}
on linuxserver2 {
device /dev/drbd0;
disk /dev/vg_linuxserver2/lv_drbd;
address 192.168.10.3:7788;
meta-disk internal;
}
}
This will create a repdata resource shared between linuxserver1 and linuxserver2. The servers IP addresses are 192.168.10.2 and 192.168.10.3 respectively. Adjust those settings to match your setup. Don’t forget to change somepassword to something more secure.
Initialize the meta-data area by running:
drbdadm create-md repdata
and start drbd:
service drbd start
check it’s status by running:
cat /proc/drbd
or:
service drbd status
As you can see , both nodes are secondary, which is normal. we need to decide which node will act as a primary now, that will initiate the first ‘full sync’ between the two nodes. Run this ONLY on the primary node:
drbdadm -- --overwrite-data-of-peer primary repdata
You can monitor the progress (on either node) of the sync by running:
watch -n 1 cat /proc/drbd
When all is done format and mount it. Run this ONLY on the primary node:
mkfs.ext4 /dev/drbd0 ; mkdir /repdata ; mount /dev/drbd0 /repdata
but don’t forget to create the mount point on the secondary node as well. Run this ONLY on the secondary node:
mkdir /repdata
Configuring Heartbeat
Heartbeat is a daemon that provides cluster infrastructure (communication and membership) services to its clients. This allows clients to know about the presence (or disappearance!) of peer processes on other machines and to easily exchange messages with them.
Let’s create the authkeys. Run:
nano /etc/ha.d/authkeys
and paste:
auth 2 2 sha1 somepassword
change somepassword to a stronger one.
Change the access rights, else heartbeat won’t start:
chmod 600 /etc/ha.d/authkeys
Create the ha.cf file, the main config file for heartbeat by running:
nano /etc/ha.d/ha.cf
and paste:
udpport 695 logfile /var/log/ha-log logfacility local0 keepalive 2 deadtime 30 warntime 10 initdead 120 bcast eth0 auto_failback on node linuxserver1 node linuxserver2
Pay close attention the the node lines, they should be the hostnames of the two linux servers. If you want to run more than one HA cluster inside the same network change the udpport to a different, unused one.
Now to the haresources file. Run:
nano /etc/ha.d/haresources
and paste:
linuxserver1 IPaddr::192.168.10.3/24 drbddisk::repdata Filesystem::/dev/drbd0::/repdata::ext4 httpd
This basically means that linuxserver1 is the primary node, the node ip address will be 192.168.10.3 on a 255.255.255.0 subnet. The filesystem is a drbddisk, namely /dev/drdb0, that should be mounted on the /repdata mountpoint as ext4. The service that should be started is httpd (apache). Adjust it to fir your needs.
And finally start the service:
service heartbeat start
IMPORTANT NOTES
- make sure that the ports listed above are open on both servers
- make sure that drbd and heartbeat start at boot time
- make sure to NOT start the service at boot time, since it is managed by heartbeat
- make sure that the service in question actually uses the /repdata mount point.
That’s it!