A load balancer removes the single point of failure from your application tier. But then the load balancer itself becomes the new single point of failure: if that one Nginx machine goes down, everything behind it becomes unreachable, no matter how many healthy app servers you have.
The standard fix is an active-standby pair: two identical Nginx load balancers sharing one virtual IP (VIP). Clients only ever talk to the VIP. Under normal conditions the master holds it; if the master dies, or just its Nginx process does, the backup claims the VIP within seconds. The protocol that makes this work is VRRP (Virtual Router Redundancy Protocol), and the software that implements it on Linux is Keepalived.
In this tutorial you will build a high availability Nginx load balancer pair with Keepalived on Ubuntu, including a real health check that fails over when the Nginx process dies (not just the whole machine), and then prove the failover works.
How VRRP Failover Works
Keepalived runs on both load balancers and they talk VRRP to each other:
- Each node has a priority. The node with the highest priority becomes MASTER and adds the virtual IP to its interface. The other node stays BACKUP, watching.
- The master sends a VRRP advertisement roughly every second: “I am alive and I hold the VIP.” If the backup stops hearing advertisements, it promotes itself and claims the VIP, announcing the change to the network with gratuitous ARP so switches update immediately. Total failover time is typically 3 to 4 seconds.
- A track script extends this beyond machine death: Keepalived runs a check command every few seconds, and if it fails (for example, Nginx crashed but the server is fine), the node drops its priority or releases mastership, triggering the same failover.
The result: the VIP is “always” up, as long as at least one of the two nodes is healthy.
Prerequisites
- Two Ubuntu 22.04 or 24.04 servers on the same subnet. VRRP works at layer 2, so both nodes must share a broadcast domain
- One unused IP on that subnet to act as the virtual IP. On your own network, just pick a free address; on a cloud provider, ask whether they support VRRP/floating IPs (many public clouds block multicast and need their own floating-IP mechanism instead)
- A user with
sudoprivileges on both servers - If a firewall runs between the nodes, VRRP (IP protocol 112) must be allowed
The Use Case
____________
| |
___________ -----> | App1 |
| | | |____________|
--> | LB M |----| ____________
| |___________| | | |
user -------- |-----> | App2 |
(VIP) | ___________ | |____________|
| | | | ____________
--> | LB B |----| | |
|___________| -----> | App3 |
|____________|
- LB M (master):
10.205.199.174/24on interfaceeth0 - LB B (backup):
10.205.199.175/24on interfaceeth0 - Virtual IP:
10.205.199.200. This is the only address users and DNS ever see
Check your real interface name with ip link show (it is more likely ens18 or enp0s3 than eth0 on modern Ubuntu) and substitute it throughout.
Step 1: Install and Configure Nginx on Both Nodes
Install Nginx from the package manager on both servers:
sudo apt update
sudo apt install nginx
Then configure it as a load balancer pointing at your app servers. The configuration must be identical on both nodes, because the backup has to be able to take over transparently. For the load balancer configuration itself, follow How to Configure Nginx as Layer 7 Load Balancer for HTTP/HTTPS balancing, or How to Configure Nginx as Layer 4 Load Balancer for TCP/UDP.
One detail that matters for HA: make sure Nginx listens on all addresses (listen 80;, which means 0.0.0.0) rather than binding to the node’s own IP. The VIP appears and disappears from the interface at failover, and a listen bound to a specific address that does not exist yet would fail.
Step 2: Install Keepalived on Both Nodes
sudo apt install keepalived
Step 3: Configure the Master
On LB M, create the configuration:
sudo nano /etc/keepalived/keepalived.conf
vrrp_script check_nginx {
script "/usr/bin/killall -0 nginx"
interval 2
fall 3
rise 2
weight -150
}
vrrp_instance VI_1 {
interface eth0
state MASTER
priority 200
virtual_router_id 33
virtual_ipaddress {
10.205.199.200
}
authentication {
auth_type PASS
auth_pass Sup3rS3cret
}
track_script {
check_nginx
}
}
What each part does:
vrrp_script check_nginx is the health check. killall -0 nginx sends signal 0 to the Nginx process: it does nothing if the process exists and fails if it does not, making it a cheap “is Nginx running?” probe. interval 2 runs it every 2 seconds; fall 3 requires 3 consecutive failures before declaring the node unhealthy (6 seconds, which avoids flapping on a blip); rise 2 requires 2 consecutive successes to declare it healthy again.
weight -150: when the check fails, subtract 150 from this node’s priority. The master drops from 200 to 50, which is below the backup’s 100, so the backup wins the next election and takes the VIP. This is how a process failure (not just machine death) triggers failover.
priority 200 must be higher than the backup’s. Highest priority holds the VIP.
virtual_router_id 33 identifies this VRRP group. Both nodes must use the same number, and it must be unique per subnet. If another team runs keepalived on the same network with ID 33, they will interfere with each other.
authentication is a shared password so random VRRP speakers cannot join your group. Both nodes must have the same one. Change it, and note VRRP passwords are truncated to 8 characters.
Enable and start Keepalived:
sudo systemctl enable --now keepalived
Step 4: Configure the Backup
On LB B, create the same file with only two differences: state BACKUP and a lower priority.
sudo nano /etc/keepalived/keepalived.conf
vrrp_script check_nginx {
script "/usr/bin/killall -0 nginx"
interval 2
fall 3
rise 2
weight -150
}
vrrp_instance VI_1 {
interface eth0
state BACKUP
priority 100
virtual_router_id 33
virtual_ipaddress {
10.205.199.200
}
authentication {
auth_type PASS
auth_pass Sup3rS3cret
}
track_script {
check_nginx
}
}
sudo systemctl enable --now keepalived
Step 5: Verify the Virtual IP
On the master, check the interface addresses:
ip addr show eth0
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP
link/ether 00:0c:29:7a:56:e5 brd ff:ff:ff:ff:ff:ff
inet 10.205.199.174/24 brd 10.205.199.255 scope global eth0
inet 10.205.199.200/32 scope global eth0
The second inet line is the VIP, so the master holds it. Run the same command on the backup: it should show only its own address. You can also watch keepalived’s state decisions live:
journalctl -u keepalived -f
Finally, hit the VIP from a client machine and confirm your load-balanced application responds:
curl http://10.205.199.200/
Step 6: Test the Failover
This is the step most tutorials skip and the one you should never skip. Kill Nginx on the master:
sudo systemctl stop nginx
Within about 6 to 8 seconds (3 failed checks plus VRRP re-election), the VIP should vanish from LB M and appear on LB B:
# on LB B
ip addr show eth0 | grep 10.205.199.200
inet 10.205.199.200/32 scope global eth0
For a more convincing demo, run a continuous request loop from a client while you stop Nginx:
while true; do curl -s -m 1 http://10.205.199.200/ -o /dev/null \
-w "%{http_code}\n"; sleep 1; done
You will see a few failed requests during the transition, then traffic resumes, now served by the backup. Start Nginx again on LB M (sudo systemctl start nginx) and the VIP moves back, because the master’s priority recovers to 200.
Common Problems and Troubleshooting
Both nodes show the VIP at the same time (split-brain).
The nodes cannot see each other’s VRRP advertisements. Causes, in order of likelihood: a firewall blocking VRRP (allow IP protocol 112 between the nodes; with UFW: sudo ufw allow to 224.0.0.18), the nodes on different subnets/VLANs, mismatched virtual_router_id or auth_pass, or a cloud network that drops multicast. On multicast-hostile networks, switch to unicast by adding these to the vrrp_instance block (with addresses swapped appropriately on each node):
unicast_src_ip 10.205.199.174
unicast_peer {
10.205.199.175
}
Failover happens on machine shutdown but not when Nginx dies.
The track script is not running or not failing. Check journalctl -u keepalived for VRRP_Script(check_nginx) failed. A common cause is the script path: use the absolute path /usr/bin/killall -0 nginx (the psmisc package provides killall; install it if missing). Newer Keepalived versions also refuse to run scripts as root unless enable_script_security conditions are met; the log will say so explicitly.
The VIP moves, but clients still get timeouts for a minute.
Something in the path cached the old MAC address. Keepalived sends gratuitous ARP on takeover, but some switches and cloud routers ignore it. Check whether your environment needs the VIP registered with the cloud provider’s own floating IP feature instead of raw VRRP.
Keepalived flaps between MASTER and BACKUP.
Usually the health check is borderline (Nginx restarting in a loop) or two VRRP groups share the same virtual_router_id on the subnet. tcpdump -i eth0 proto 112 shows every VRRP speaker on the wire, including ones that are not yours.
Best Practices
Keep both Nginx configs in sync, automatically. A backup that takes over with stale configuration is a subtle outage. Sync /etc/nginx with rsync in a cron job, or manage both nodes from one playbook. See Getting Started with Ansible on Ubuntu.
Consider nopreempt for production. By default, when the old master recovers it preempts, meaning it takes the VIP back, causing a second traffic blip. Adding nopreempt to both vrrp_instance blocks (and setting both state BACKUP) means whoever currently holds the VIP keeps it until it actually fails, halving your failover events.
Use a real HTTP check for deeper health. killall -0 only proves the process exists. If you want failover when Nginx is up but misbehaving, point the script at curl -sf http://localhost/health || exit 1 instead.
Monitor the standby too. The classic HA failure: the master dies after the backup has been silently broken for weeks. Alert on keepalived state transitions and on the backup’s own Nginx health, not just the VIP.
Conclusion
You now have an active-standby Nginx load balancer pair: Keepalived elects a master by priority, the check_nginx track script demotes a node the moment its Nginx process dies, and the virtual IP follows the healthy node with a few seconds of disruption. You verified all of it by actually pulling the trigger, which is the only failover test that counts.
To go deeper on the pieces: How to Install and Configure Keepalived on Ubuntu covers VRRP, priorities, and preemption in isolation; How to Configure Nginx as Layer 4 Load Balancer and How to Configure Nginx as Layer 7 Load Balancer cover the balancing tier itself; and once the pair is live, load test the VIP with Load Testing with k6 on Ubuntu to confirm the failover behaviour under real traffic.