How to build a High Availability Apache Cluster on Centos 7

The Project

One of our clients recently approached us to deploy a High Availability (HA) solution for a mission-critical website. The website required LAMP (Linux, Apache, MySQL and PHP) stack capable of withstanding a high volume of traffic and as close to 100% uptime as possible.

After some research, we decided the best way to implement this solution within a reasonable budget would be to use two dedicated servers, each running an almost identical setup of two separate Virtual Machine (VM) instances. Each dedicated server would host a VM for Apache and a separate one for MySQL.

The pair of Apache VMs would share a single floating IP despite being on different hardware and the same goes for the MySQL VMs. That way if either dedicated server went offline for any reason, the other could pick up the slack.

That was the concept, what follows is brief overview of how we implemented it.

Software

We utilised Proxmox 4 to provide the backend virtualization platform. There are many alternatives but this is a free system that allow easy deployment, management and backup tools. The ability to take snapshots and rollback quickly was very important to us, especially during testing. Proxmox also allowed us to use KVM as our virtualization technology.

All of the software used in this solution is freely available and open source.

Initial Setup

Deploying Proxmox is outside the scope of this article but it was a standard setup. Once installed on both dedicated servers, we moved onto deploying a CentOS 7 Virtual Machine on each Proxmox host. We named them apache1 and apache2 for simplicity.

Please note we will shortly be updating this article to include a guide to creating each MySQL VM and also for setting up a higher performance build of Apache using PHP-FPM.

Setup Guide

In this guide there will be some commands which need run on both nodes and some just run on a single node. Commands will be labelled “both“, “apache1” or “apache2“.

We are also using a simple IP structure which could be improved on if needed including running the services over different network adapters etc. For now we’re focusing on simplicity.

Step 1 – Setup name resolution:

Please note the IP addresses in this guide are picked at random as follows:

  • apache1 – 91.91.0.1
  • apache 2 – 91.91.0.2
  • floating IP – 91.91.0.3

We need to make sure both hosts can talk with each other so on BOTH nodes I ran the following commands.

[code type=codetype]

## BOTH:

nano /etc/hosts

91.91.0.1 apache1
91.91.0.2 apache2

[/code]

STEP 2 – configure firewall:

At this stage we are going to install firewalld rather than use iptables. This might already be loaded in your system otherwise install it using yum.

[code type=codetype]

## BOTH:

# Check if the firewall is running:

firewall-cmd –state

# If its not running install it and active it:

yum install -y firewalld
systemctl start firewalld.service
systemctl enable firewalld.service

# Lets load the High – Availability service into the firewall:

firewall-cmd –permanent –add-service=high-availability

# Then we reload it:

firewall-cmd –reload

[/code]

STEP 3 – Installing pacemaker (PCS Service):

We are going to install the required services from yum.

[code type=codetype]

## BOTH:

# Install PCS

yum install -y pcs

# Disable SELinux

nano /etc/sysconfig/selinux

# Change “SELINUX=enforcing” to “SELINUX=disabled” and save the configuration file

SELINUX=disabled

# Reboot both systems

reboot

# Setup a password for hacluster

echo “yourpassword” | passwd hacluster –stdin

# Start and enable the pcs service:

systemctl start pcsd.service
systemctl enable pcsd.service

[/code]

STEP 4 – Joining the services:

At this point we are going to join the cluster service up.

[code type=codetype]

## Apache1

# Lets join up the services:

pcs cluster auth apache1 apache2 -u hacluster -p yourpassword

# Next, we’ll generate and synchronize the Corosync configuration. Here, we’ll name the cluster cluster, but you can call it anything

pcs cluster setup –name cluster apache1 apache2

[/code]

STEP 5 – starting THE SERVICES:

We are going to start the services now and enable them so they startup on reboot.

[code type=codetype]

## Apache1

# Starting the services:

pcs cluster start –all

## BOTH

# Enable the services on both sides:

systemctl enable corosync.service
systemctl enable pacemaker.service

# Check its running and worked:

pcs status

[/code]

STEP 6 – Disable STONITH:

As we are only running a two node cluster quorum will not function correctly. As such we require to disable it.
[code type=codetype]

## Apache 1

pcs property set stonith-enabled=false
pcs property set no-quorum-policy=ignore

[/code]

STEP 7 – install apache:

During this stage I am only going to install Apache and nothing else as in my next guide I am going to explain how to setup a customised build of Apache with PHP-FPM.

[code type=codetype]

## BOTH

# We are going to install Apache and also lynx. Lynx is optional but lets us test the service status is working correctly.

yum install -y httpd lynx

nano /etc/httpd/conf.d/status.conf

# Add the following code:

<Location /server-status>
SetHandler server-status
Order Deny,Allow
Deny from all
Allow from 127.0.0.1
</Location>

# Restart Apache:

systemctl restart httpd

# Test the page works:

lynx 127.0.0.1//server-status

# Allow Apache on the firewall:
firewall-cmd –permanent –add-service=http
firewall-cmd –permanent –add-service=https
firewall-cmd –reload
[/code]

STEP 8 – Setting up the virtual ip:

At this point we are ready to setup the floating IP which will the the IP on which Apache will respond to requests. The system is going to be setup using a monitoring interval of 10 seconds.

[code type=codetype]

## Apache 1

# During this command you will need to set your own IP address, for our example the IP will be: 91.91.0.3

pcs resource create VIP ocf:heartbeat:IPaddr2 ip=91.91.0.3 cidr_netmask=24 op monitor interval=10s
pcs resource defaults resource-stickiness=100

# To test it works run:

pcs status

# Then if your pcs status shows it started on Apache1 as it does in my example, check you can see the IP on the network adapter:

ip addr

[/code]

STEP 9 – Adding apache as a cluster service:

We now require to add the apache service to the cluster and also configure the cluster so that Apache and the IP address are within the same resource group as we don’t want Apache running on one server and the IP on another.

[code type=codetype]

## Apache 1

# Adding Apache to PCS

pcs resource create website ocf:heartbeat:apache configfile=/etc/httpd/conf/httpd.conf statusurl=”http://localhost/server-status” op monitor interval=10s

# Create a group which will include Apache and the IP Address so that they stay bound on the same server:

pcs resource group add webresource VIP website

# You can then check its all working correctly, be aware if you find it won’t start I have found rebooting both systems can help.

pcs resource show
pcs status
[/code]

STEP 10 – How to restart apache:

Please be aware you can’t restart Apache using your normal method. You must restart apache using the pcs command which based on this setup would be:

[code type=codetype]
pcs resource restart website
[/code]

It can also be helpful for testing to be able to test apache outside of the pcs service and to disable it use:

[code type=codetype]
pcs resource disable website

# When done:

pcs resource enable website
[/code]

Conclusion:

Using this guide you should have now setup two servers both running Apache in and Active-Passive Cluster with a Floating IP. This means if you can place two servers in different diverse locations you can create a more fault tolerant setup. This guide does not go into detail about how we sync files between the servers but for Apache we will use rsync as we do not feel DRDB is required but we will use this when we setup the MySQL Active-Passive setup shortly.

I would welcome any feedback or questions you might have on this guide.