Install and Configure HAProxy Load Balancer on Ubuntu

One server is a single point of failure, no matter how good the server is. The moment you run two copies of an application, you need something in front of them that spreads the traffic and, more importantly, stops sending requests to a copy that just died. That something is a load balancer, and HAProxy is one of the best ever written: free, absurdly efficient, and running in front of a large share of the internet’s busiest sites.

This tutorial teaches you HAProxy from zero on Ubuntu. You will install it, load balance HTTP traffic across two backend servers in layer 7 mode, watch health checks eject a dead server automatically, monitor everything through the built-in stats dashboard, and see how the same tool balances raw TCP (databases, message queues, Kubernetes API servers) in layer 4 mode.

This guide is for sysadmins and DevOps engineers with basic Linux skills. No prior load balancing experience is needed.

The Concepts: Frontends, Backends, and the Two Layers

HAProxy configuration is built from two main building blocks, and once they click, every config file you will ever read makes sense:

• A frontend is where traffic enters: an IP and port HAProxy listens on, plus rules about what to do with connections that arrive.
• A backend is a named pool of real servers that can serve that traffic, plus the strategy for choosing among them and the health checks that decide who is eligible.

A frontend points at a backend. That is the whole model.

The other decision is which layer to operate on:

• Layer 4 (mode tcp) forwards TCP connections without understanding their contents. It is protocol-agnostic and extremely fast: MySQL, Redis, SMTP, or TLS traffic you do not want to decrypt, all work.
• Layer 7 (mode http) parses HTTP. HAProxy can then route by path or header, add headers like X-Forwarded-For, retry failed requests, and run smarter health checks that verify the application actually answers, not just that the port opens.

The rule of thumb: use mode http for web traffic, mode tcp for everything else.

Prerequisites

1. Three servers or VMs with Ubuntu 22.04 or 24.04: one for HAProxy (lb, 10.0.0.10) and two web backends (web1, 10.0.0.21 and web2, 10.0.0.22). You can follow along with less by running backends on different ports of one machine.
2. A user with sudo privileges on all of them
3. Basic command line knowledge

Step 1: Prepare Two Backend Web Servers

We need something to balance. On each backend, install nginx and serve a page that identifies the server, so we can literally see the load balancing work.

On web1:

sudo apt update && sudo apt install nginx -y
echo "response from web1" | sudo tee /var/www/html/index.html

On web2:

sudo apt update && sudo apt install nginx -y
echo "response from web2" | sudo tee /var/www/html/index.html

Verify both from the load balancer machine:

curl http://10.0.0.21 && curl http://10.0.0.22

response from web1
response from web2

Step 2: Install HAProxy

On the lb machine:

sudo apt update
sudo apt install haproxy -y
haproxy -v

HAProxy version 2.8.5-1ubuntu3 2024/04/01 - https://haproxy.org/

Ubuntu’s package is a stable LTS branch of HAProxy and is enabled as a systemd service automatically. Everything we configure lives in a single file: /etc/haproxy/haproxy.cfg.

Step 3: Configure Layer 7 HTTP Load Balancing

Open the config:

sudo nano /etc/haproxy/haproxy.cfg

The file already contains global (process-level settings: logging, limits, the unprivileged user it drops to) and defaults (settings inherited by every frontend and backend, mostly timeouts). Leave both as shipped, they are sensible. Append your first frontend and backend at the bottom:

frontend web-frontend
    bind *:80
    mode http
    option httplog
    default_backend web-backend

backend web-backend
    mode http
    balance roundrobin
    option httpchk GET /
    http-check expect status 200
    default-server inter 3s fall 2 rise 2
    server web1 10.0.0.21:80 check
    server web2 10.0.0.22:80 check

Line by line:

• bind *:80 listens on port 80 of every address on this machine.
• option httplog enables rich per-request logging to /var/log/haproxy.log.
• balance roundrobin alternates requests across servers. Other useful algorithms: leastconn (best when request durations vary a lot) and source (same client IP goes to the same server, a poor man’s sticky session).
• option httpchk GET / with http-check expect status 200 is a layer 7 health check: instead of just opening a TCP connection, HAProxy sends a real HTTP request every 3 seconds (inter 3s) and demands a 200. A backend whose nginx runs but serves 500s is correctly treated as dead.
• fall 2 marks a server down after 2 consecutive failures; rise 2 requires 2 consecutive successes to bring it back.
• Each server line is a pool member; check turns its health checking on.

Validate the file, then apply:

sudo haproxy -c -f /etc/haproxy/haproxy.cfg

Configuration file is valid

sudo systemctl reload haproxy

Use reload rather than restart from now on: HAProxy performs a graceful handover where the old process finishes its active connections while the new one accepts fresh ones. Zero dropped requests.

Step 4: Watch It Balance and Fail Over

From any machine, hit the load balancer a few times:

curl http://10.0.0.10
curl http://10.0.0.10
curl http://10.0.0.10

response from web1
response from web2
response from web1

Round robin, exactly as configured. Now the important part, the reason load balancers exist. Kill one backend:

# on web2
sudo systemctl stop nginx

Within about six seconds (two failed checks, 3 seconds apart), HAProxy ejects it. Keep curling the load balancer:

response from web1
response from web1
response from web1

No errors, no timeouts, every request lands on the survivor. Check the log on the lb machine to see the state change:

sudo grep DOWN /var/log/haproxy.log

Server web-backend/web2 is DOWN, reason: Layer7 timeout, check duration: 3000ms. 1 active and 0 backup servers left.

Start nginx again on web2 and it re-enters the pool automatically after two clean checks. Nobody had to touch the load balancer, and that is the point.

Step 5: Enable the Stats Dashboard

HAProxy ships a live dashboard showing every frontend, backend, and server with its state, check results, and traffic counters. Add this to the bottom of the config:

listen stats
    bind *:8404
    mode http
    stats enable
    stats uri /stats
    stats refresh 10s
    stats auth admin:StrongPasswordHere

(listen is a shorthand block that is frontend and backend in one.) Reload:

sudo haproxy -c -f /etc/haproxy/haproxy.cfg && sudo systemctl reload haproxy

Open http://10.0.0.10:8404/stats in a browser and log in. Green rows are healthy servers, red rows are down, and the columns show sessions, bytes, and check status in real time. During the failover test above, you would have watched web2 turn red and back to green.

Do not expose this port to the internet: firewall it to your admin network, and use a real password.

Step 6: Layer 4 Mode for Non-HTTP Traffic

The same tool balances any TCP service by switching modes. Here is a complete example fronting two PostgreSQL replicas:

frontend postgres-frontend
    bind *:5432
    mode tcp
    option tcplog
    default_backend postgres-backend

backend postgres-backend
    mode tcp
    balance leastconn
    option tcp-check
    default-server inter 5s fall 2 rise 2
    server pg1 10.0.0.31:5432 check
    server pg2 10.0.0.32:5432 check

Differences from the HTTP setup: mode tcp in both blocks, option tcp-check (a connection test instead of an HTTP request), and leastconn, which suits long-lived database connections better than round robin.

This layer 4 pattern is exactly how you put HAProxy in front of a Kubernetes control plane, which I covered as its own tutorial: How To Install HAProxy for Kubernetes Load Balancer.

One caution: in TCP mode the backend sees connections coming from the load balancer’s IP, not the client’s. For databases that is usually fine. For HTTP, it is another reason to prefer mode http, where HAProxy adds X-Forwarded-For automatically when you add option forwardfor to the defaults or backend.

Common Mistakes and Troubleshooting

haproxy -c complains about a missing bind or crashes on start. Another process already owns the port (a leftover nginx on the lb machine is the classic). Check with sudo ss -tlnp | grep :80.

All servers marked DOWN immediately. From the lb machine, test each backend directly with curl http://10.0.0.21. If that works but checks fail, your health check is stricter than the app: for example http-check expect status 200 while the app returns a 301 redirect on /. Point httpchk at a real health endpoint.

503 Service Unavailable from HAProxy. No healthy server in the backend. This is HAProxy working correctly, look at the stats page or the log to learn why the servers are failing checks.

Uneven balancing. With very few test requests, browser connection reuse skews results, one browser tab may ride a single connection to a single server. Test with curl in a loop, not by refreshing a browser.

Everything works until a config edit, then reload silently keeps old behavior. The reload failed validation and systemd kept the old process. Always run haproxy -c -f /etc/haproxy/haproxy.cfg before reloading, and check sudo systemctl status haproxy after.

Backends log the load balancer’s IP for every client. In HTTP mode, add option forwardfor to the backend and configure the web servers to read X-Forwarded-For. In TCP mode this is expected behavior.

Best Practices

1. Health check the application, not the port. A /health endpoint that verifies database connectivity beats GET /, and both beat a bare TCP check for HTTP services.
2. Always reload, never restart, and always validate with -c first. Graceful reloads are one of HAProxy’s superpowers; use them.
3. Match the algorithm to the workload: roundrobin for uniform short requests, leastconn for long or uneven ones, source only when you truly need stickiness.
4. Protect and monitor the stats page, and consider scraping HAProxy metrics into Prometheus once you run it seriously: Setup Prometheus and Grafana on Ubuntu.
5. The load balancer itself is now your single point of failure. The fix is a second HAProxy node with a floating virtual IP managed by keepalived, which I covered step by step in Install and Configure keepalived on Ubuntu.
6. Load test before production so you know your real numbers for connections and requests per second: Load Testing with k6 on Ubuntu.

Conclusion

You installed HAProxy on Ubuntu and built both of its fundamental setups: a layer 7 HTTP load balancer with real application health checks, automatic failover you tested with your own hands, and a live stats dashboard, plus a layer 4 TCP pool for everything that is not HTTP. The frontend/backend model you learned here reads the same in every HAProxy config you will ever encounter, from a two-server blog to a Kubernetes control plane.

Natural next steps: make the load balancer itself highly available with keepalived, or apply the TCP pattern to your cluster with HAProxy for Kubernetes Load Balancer. And if you are curious how the nginx alternative compares, see Configure Nginx as Layer 7 Load Balancer.