Nginx can load balance at two different layers, and picking the right one matters. A Layer 7 load balancer understands HTTP: it can route by URL, inspect headers, and terminate TLS. A Layer 4 load balancer works one level down, on raw TCP and UDP connections: it never looks inside the traffic, it just forwards bytes. That makes it protocol-agnostic (databases, MQTT, game servers, SMTP, TLS-passthrough HTTPS, anything), lighter on CPU, and simpler to reason about.
In this tutorial you will configure Nginx as a Layer 4 load balancer using the stream module: balance TCP traffic across three application nodes, understand the health checking and balancing methods available, and verify traffic is actually being distributed. If you need HTTP-aware routing instead, see How to Configure Nginx as Layer 7 Load Balancer.
Prerequisites
- A server running Ubuntu 22.04 or 24.04 LTS for the load balancer
- Nginx installed.
sudo apt install nginxis enough; Ubuntu’s package ships the stream module - Two or more backend application nodes reachable from the load balancer
- A user with
sudoprivileges
The Use Case
____________
| |
-----> | App1 |
| |____________|
___________ | ____________
| | | | |
user -----------> | LB |----|-----> | App2 |
|___________| | |____________|
| ____________
| | |
-----> | App3 |
|____________|
- LB node:
10.11.12.13, listening on port3000 - App1:
10.1.1.10:4000· App2:10.1.1.20:4000· App3:10.1.1.30:4000
The application speaks TCP; if it uses TLS/HTTPS, the encryption passes straight through and is terminated at the app nodes. A Layer 4 balancer forwards the encrypted bytes without ever decrypting them. (If you want the load balancer itself to terminate TLS at layer 4, that is possible too. See Nginx Layer 4 Load Balancer with SSL.)
Step 1: Confirm the Stream Module Is Available
The Layer 4 functionality lives in the stream module. Check how your Nginx was built:
nginx -V 2>&1 | tr ' ' '\n' | grep stream
--with-stream=dynamic
--with-stream_ssl_module
Three possible outcomes:
--with-stream=dynamic(Ubuntu’s package): the module exists as a loadable.sofile and must be loaded with aload_moduledirective. On Ubuntu, installinglibnginx-mod-stream(usually pulled in by thenginxmetapackage) provides it.--with-stream(static): the module is compiled in; noload_moduleneeded.- No output: your build has no stream support. Build from source with
--with-stream; see How to Install Nginx from Source.
Step 2: Create the Base Configuration
Back up the default configuration, then open a fresh one:
sudo cp /etc/nginx/nginx.conf /etc/nginx/nginx.conf.old
sudo nano /etc/nginx/nginx.conf
Start with the global settings:
user www-data;
worker_processes auto;
worker_rlimit_nofile 8192;
pid /run/nginx.pid;
load_module /usr/lib/nginx/modules/ngx_stream_module.so;
events {
worker_connections 4096;
}
If your stream module is dynamic (the Ubuntu default), the load_module line is required. If it is compiled in statically, delete that line, because loading an already-built-in module is an error.
worker_rlimit_nofile and worker_connections set the connection capacity. Remember every proxied connection consumes two (client side and upstream side). These defaults are fine to start; tune them based on a load test, not guesswork. Load Testing with k6 on Ubuntu shows how.
Step 3: Add the Stream Block
Below the events block (not inside it, and not inside an http block), add the Layer 4 configuration:
stream {
upstream app_node {
server 10.1.1.10:4000 max_fails=2 fail_timeout=10s;
server 10.1.1.20:4000 max_fails=2 fail_timeout=10s;
server 10.1.1.30:4000 max_fails=2 fail_timeout=10s;
}
server {
listen 3000;
proxy_pass app_node;
proxy_connect_timeout 3s;
proxy_timeout 10m;
}
}
What each part does:
The upstream block lists the backends. By default Nginx distributes new connections round robin: first connection to App1, second to App2, and so on.
max_fails and fail_timeout configure the passive health check. With max_fails=2 fail_timeout=10s, if connections to a backend fail twice within 10 seconds, Nginx marks it unavailable for the next 10 seconds and sends traffic to the others. This is passive: Nginx learns from real connection failures; open-source Nginx has no active TCP probes (that is an Nginx Plus feature). Passive checks are usually enough, since a dead backend is detected within a couple of failed connections.
proxy_connect_timeout is how long to wait when establishing the connection to a backend before counting it as a failure and trying the next one. Keep it short (a few seconds) so clients fail over quickly.
proxy_timeout is how long an established connection may sit idle before Nginx closes both sides. The default is 10 minutes; raise it for long-lived protocols (database connections, MQTT).
No TLS configuration here. For TLS passthrough, the encrypted bytes flow through untouched, and certificates live on the app nodes.
Balancing methods other than round robin
Add one directive at the top of the upstream block to change the algorithm:
upstream app_node {
least_conn; # send new connections to the least-busy backend
...
}
least_connpicks the backend with the fewest active connections. Better than round robin when connection lifetimes vary a lot.hash $remote_addr consistent;gives session affinity: the same client IP always lands on the same backend. Use it when backends keep per-client state.
UDP load balancing
The stream module balances UDP the same way. Add udp to the listen directive:
server {
listen 3000 udp;
proxy_pass app_node;
}
This works for DNS, syslog, RTP, and game traffic. Since UDP has no connections, “balancing” happens per datagram (or per session with proxy_responses tuning).
Step 4: Test and Reload
sudo nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
sudo systemctl reload nginx
Step 5: Verify the Load Balancing
Confirm Nginx is listening on the frontend port:
sudo ss -ntlp | grep 3000
Then hit the load balancer repeatedly from a client. If the app is HTTP(S) behind the L4 balancer, something like this makes the distribution visible (assuming each app node reports its identity, e.g. via hostname in a header or response):
for i in $(seq 1 6); do curl -s http://10.11.12.13:3000/whoami; done
app1
app2
app3
app1
app2
app3
Round robin in action. Now prove the failover: stop the app on one node and repeat. After a failed attempt or two, all traffic flows to the remaining backends, and the dead node stops appearing.
You can also watch connections land on the backends in real time from an app node:
sudo ss -ntp | grep 4000
Common Problems and Troubleshooting
nginx: [emerg] "stream" directive is not allowed here.
The stream block is inside the http block, or the config that Ubuntu splits across /etc/nginx/conf.d/ got included at the wrong level. stream must sit at the top level of nginx.conf, sibling to events and http.
nginx: [emerg] unknown directive "stream".
The stream module is not loaded. Add the load_module line from Step 2 (and install libnginx-mod-stream if the .so file does not exist).
nginx: [emerg] module is already loaded.
Your build has the module compiled in statically, or Ubuntu auto-loads it via /etc/nginx/modules-enabled/. Remove your manual load_module line.
Clients connect but get dropped after exactly 10 minutes.
That is proxy_timeout closing idle connections. Raise it, or make the application send keepalives more often than the timeout.
One backend gets all the traffic.
With hash $remote_addr, all requests from one test machine will hit one backend. That is the affinity working as designed. Test from multiple source IPs, or switch to round robin/least_conn for the test.
The backend sees the load balancer’s IP, not the client’s.
Expected at Layer 4: the balancer opens its own connection to the backend. If the backend must know the real client IP, enable proxy_protocol on; in the server block. But note that the backend application must also be configured to parse the PROXY protocol header, otherwise it will reject the connections as garbage.
Best Practices
Always set max_fails/fail_timeout explicitly. The defaults (1 failure / 10 seconds) are aggressive: a single timeout takes a healthy-but-momentarily-slow backend out of rotation. Two or three failures within the window is a more stable signal.
Keep proxy_connect_timeout short and proxy_timeout matched to your protocol. Connect timeouts are about failover speed; idle timeouts are about protocol semantics. They solve different problems, so do not tune one to fix the other.
Make the load balancer itself redundant. A single L4 balancer is a single point of failure. Pair it with a second node and a floating virtual IP: Setup a High Availability Load Balancer with Nginx and Keepalived on Ubuntu.
One balancer can front many services. Multiple server blocks with different listen ports (and different upstreams) coexist in one stream block. You can even route multiple TLS domains on one port using SNI. See Multi Domain Configuration in Layer 4 Nginx Load Balancer.
Conclusion
You have configured Nginx as a Layer 4 load balancer: the stream module distributes raw TCP (or UDP) connections across your backends round robin, passive health checks pull dead nodes out of rotation, and timeouts control both failover speed and idle-connection behaviour, all without Nginx ever inspecting the traffic itself.
From here: terminate TLS at the balancer with Nginx Layer 4 Load Balancer with SSL, serve multiple domains through one balancer with Multi Domain Configuration in Layer 4 Nginx Load Balancer, and remove the balancer as a single point of failure with Nginx and Keepalived.