How to Set Up NGINX Load Balancing Across Multiple Servers

To load-balance several application servers with NGINX, define an upstream group in the http context, list each backend, and send a public location block’s proxy_pass directive to that group. Round robin is the default HTTP method. You can then choose least connections or a hash method, configure passive failure handling, and test the behavior before putting the proxy into production.

Antihost offers Linux VPS plans with full root access, unlimited traffic on a 1 Gbps port and a 14-day free trial.

This guide shows a complete configuration, explains which method fits different workloads, distinguishes open-source passive checks from NGINX Plus active checks, and covers sessions, shared upstream state, TCP/UDP services, rollout checks, and common failure symptoms.

#	Preview	Product	Price
1		NGINX Cookbook: Advanced Recipes for High-Performance Load Balancing	$37.53	Buy on Amazon
2		NGINX Cookbook: Over 70 recipes for real-world configuration, deployment, and performance	$23.69	Buy on Amazon
3		NGINX Unit Cookbook: Recipes for Using a Versatile Open Source Server	$12.31	Buy on Amazon
4		NGINX HTTP Server: Harness the power of NGINX with a series of detailed tutorials and real-life...	$51.99	Buy on Amazon
5		NGINX Cookbook: Advanced Recipes for High-Performance Load Balancing	$26.99	Buy on Amazon

As an Amazon Associate I earn from qualifying purchases.

The basic NGINX load-balancer design

NGINX sits at the public address as a reverse proxy. Clients connect to NGINX; NGINX selects one server from an upstream group and forwards the request. The application servers can remain on private addresses and do not need to be exposed directly to the internet.

An upstream group is reusable. Every proxy location that references the group can use the same backend pool, while the balancing algorithm and server parameters stay in one place. The public-facing server block handles listening and routing; the upstream block handles membership and selection.

#1 Best Overall

Sale

NGINX Cookbook: Advanced Recipes for High-Performance Load Balancing

$37.53

Buy on Amazon

Minimal HTTP configuration

Place the upstream definition and the server block inside the http context of your NGINX configuration. Replace the hostnames and port with addresses reachable from the NGINX machine.

http {
    upstream app_backend {
        server app1.example.com:8080;
        server app2.example.com:8080;
        server app3.example.com:8080;
    }

    server {
        listen 80;

        location / {
            proxy_pass http://app_backend;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto $scheme;
        }
    }
}

The name app_backend is local to this configuration; it is not a DNS record. The value in proxy_pass must match that upstream name. The proxy headers shown are a conventional starting point: your application may require different trust rules or additional headers, so validate them against the framework and authentication layer you run.

What happens to a request

1. The client resolves your public hostname and opens a connection to NGINX.
2. NGINX matches the request to a server block and then a location.
3. The upstream name in proxy_pass selects the backend group.
4. The configured method chooses one available server, taking weights into account where the method supports them.
5. NGINX forwards the request and returns the selected server’s response to the client.

Adding capacity and controlling traffic share

Add one server line for each backend. Every address must be reachable from the NGINX host, and every backend should serve the same application contract: routes, authentication behavior, response format, and health endpoint.

Weighted distribution

By default, NGINX distributes HTTP requests with round robin. A server’s weight changes its relative share rather than setting an absolute request count.

upstream app_backend {
    server app1.example.com:8080 weight=3;
    server app2.example.com:8080 weight=1;
    server app3.example.com:8080 weight=1;
}

In this example, app1 is offered roughly three times the traffic of either app2 or app3 when all servers are available and requests are comparable. Weight is useful when machines have different capacity, but it does not compensate for an application that has a slower dependency or a different failure rate.

Rank #2

Sale

NGINX Cookbook: Over 70 recipes for real-world configuration, deployment, and performance

$23.69

Buy on Amazon

Taking a server out of rotation

For a planned drain, remove the server from the upstream configuration and reload NGINX after validating the file. If you need to keep the line for a later return, use the server-state controls appropriate to your NGINX edition and deployment process. Always confirm how existing long-lived connections are handled before maintenance; a configuration reload does not turn an established application connection into a new request.

Choosing the HTTP balancing method

There is no universally best algorithm. Choose according to request duration, backend capacity, cache locality, and whether the application has a shared state model.

Round robin for uniform application servers

Round robin is the default HTTP behavior. NGINX cycles through available servers while considering configured weights. It is a sensible first choice when servers have similar CPU and memory capacity and requests have similar cost.

Least connections for uneven request duration

Enable it inside the upstream block:

upstream app_backend {
    least_conn;
    server app1.example.com:8080;
    server app2.example.com:8080;
    server app3.example.com:8080;
}

NGINX favors the server with the fewest active connections, with weights still taken into account. This can reduce queueing when one request holds a connection much longer than another. It is not a latency monitor: a server can have few connections and still be slow because of a database or external service.

IP hash for client affinity

upstream app_backend {
    ip_hash;
    server app1.example.com:8080;
    server app2.example.com:8080;
    server app3.example.com:8080;
}

IP hashing derives the selection from the client IP and keeps that client on the same server while the selected server remains available. It can help legacy applications that store a session in local memory, but it is only an affinity mechanism. Mobile networks, corporate proxies, and carrier NAT can change or collapse client IPs, and a failed server still causes the client to move.

Rank #3

Sale

NGINX Unit Cookbook: Recipes for Using a Versatile Open Source Server

$12.31

Buy on Amazon

Generic hashing and cache locality

upstream cache_workers {
    hash $request_uri consistent;
    server cache1.example.com:8080;
    server cache2.example.com:8080;
    server cache3.example.com:8080;
}

Here the URI is the key, so identical paths tend to reach the same backend. The consistent option limits remapping when members are added or removed, which helps cache-like workloads. Do not use a URI hash as a substitute for user-session affinity unless the application’s state model actually matches the URI key.

Sessions, carts, and other application state

Round robin and least connections can send successive requests from one browser to different servers. If login sessions, shopping carts, upload state, or other data live only in a backend process, the user may appear logged out or lose data after a rebalance.

Prefer shared state where possible

A shared session store or another replicated state design lets any backend handle the next request. This is generally more resilient than relying on one client always returning to one machine, because a server can be replaced without taking every local session with it.

Use affinity as a deliberate compromise

IP hashing can keep a client on one backend while it is healthy. Document the trade-off: traffic may become uneven, NAT users may share one hash identity, and failover moves the client to another server. NGINX Plus also documents additional sticky-session persistence, but stickiness still does not replicate data between application servers.

Test the actual state your application uses. A simple anonymous page can work while login cookies, WebSockets, carts, or multipart uploads fail when requests change backends.

Failure detection and removing unhealthy backends

Passive behavior in open-source NGINX and NGINX Plus

Both editions can react to failures observed during live transactions. When a backend repeatedly fails according to its configured failure parameters, NGINX can mark it unavailable temporarily and stop sending new requests to it. This is passive checking: a real client request must reach the server and produce a failure before NGINX learns that the server is unhealthy.

Rank #4

NGINX HTTP Server: Harness the power of NGINX with a series of detailed tutorials and real-life examples

$51.99

Buy on Amazon

Choose failure parameters that match your request timeouts and traffic volume. A setting that is too aggressive can eject a server because of a brief dependency outage; a setting that is too lenient can send many users to a broken backend before it is removed. Watch error logs and upstream response metrics after changing them.

Active HTTP checks with NGINX Plus

NGINX Plus can send periodic health-check requests independently of user traffic and evaluate the response. This can detect a failed application before the next customer request arrives. The commercial health-check module requires the upstream group to reside in shared memory.

Expose a dedicated lightweight endpoint such as /healthz only when your application defines it. A useful endpoint checks the dependencies that matter for serving traffic, such as a database or queue, rather than merely proving that a process accepts TCP connections. Keep the endpoint fast and ensure it does not mutate data.

Recovery and re-entry

Health handling is not complete until you verify recovery. After repairing a backend, confirm that NGINX allows it back into rotation, then send requests that exercise the application rather than only checking that the port is open. If the server repeatedly leaves and re-enters rotation, investigate application logs, dependency latency, connection limits, and the health-check path instead of simply increasing thresholds.

Shared upstream zones and worker coordination

NGINX workers normally keep their own copy of upstream state. An upstream zone places the server-group configuration and related counters in shared memory so workers can coordinate.

Best Value

NGINX Cookbook: Advanced Recipes for High-Performance Load Balancing

$26.99

Buy on Amazon

http {
    upstream app_backend {
        zone app_backend 64k;
        least_conn;
        server app1.example.com:8080;
        server app2.example.com:8080;
        server app3.example.com:8080;
    }

    server {
        listen 80;
        location / {
            proxy_pass http://app_backend;
        }
    }
}

The size shown is an example allocation; choose a value appropriate for the number of members and features you use. A shared zone matters most when accurate cross-worker counters are important. The official guidance notes that least-connections selection can be less accurate at low load without a shared zone. Zones also enable dynamic upstream configuration scenarios supported by the edition and modules you run.

A zone does not make application data shared. It coordinates NGINX’s view of the upstream group; sessions, caches, and business records still need their own storage or replication strategy.

HTTP and HTTPS versus TCP and UDP services

HTTP and HTTPS

Use the http context and proxy_pass for HTTP traffic. TLS can terminate at NGINX or remain between NGINX and the backend according to your security design. If the application needs the original scheme, host, or client address, pass and validate the corresponding forwarding headers.

Raw TCP and UDP

For protocols that are not HTTP, use the stream context. The upstream syntax is similar, but routing is connection-oriented and there is no HTTP URI or header to proxy.

stream {
    upstream game_service {
        server game1.example.com:9000;
        server game2.example.com:9000;
    }

    server {
        listen 9000;
        proxy_pass game_service;
    }
}

NGINX documents weighted round robin as the default stream method and also supports least connections, least time, and hash selection for stream upstreams. Select a method based on connection duration and protocol behavior; an HTTP request-count assumption does not automatically apply to a long-lived TCP connection.

Safe deployment procedure

Use this sequence for a new pool or a change to an existing one. Perform it from the NGINX host or through your normal controlled deployment system.

1. Confirm addresses and ports. From the NGINX host, verify that every backend address resolves as expected and that the configured port is reachable through local firewalls, security groups, and network routes.
2. Check the application response. Request the real application path from each backend directly where policy permits. Verify status codes, required headers, authentication behavior, and the dedicated health endpoint if one exists.
3. Define the upstream group. Add one server entry per backend. Choose round robin, least connections, or a hash method deliberately; add weights only when capacity differences justify them.
4. Wire the public location. Set proxy_pass to the upstream name and configure forwarding headers that your application trusts. Check WebSocket or streaming requirements separately if the service uses long-lived connections.
5. Validate syntax. Run nginx -t against the intended configuration. Do not reload if the test reports an error; fix the referenced file and line first.
6. Reload without replacing the process. Use your operating system’s controlled NGINX reload mechanism after the syntax test succeeds. Keep the previous configuration available for a quick rollback.
7. Generate repeated requests. Send enough requests to observe more than one backend. Inspect access logs and upstream variables or application logs to confirm the intended distribution.
8. Simulate one failure. Stop or isolate one backend in a controlled window. Verify that passive behavior stops assigning new requests after the configured failures, or that an active check marks it unavailable when using NGINX Plus.
9. Restore and observe recovery. Bring the backend back, confirm successful health behavior, and watch for repeated failure and recovery cycles.
10. Exercise stateful paths. Test login, logout, carts, uploads, WebSockets, and any operation that depends on connection or session affinity. Anonymous requests alone are insufficient.
11. Monitor after rollout. Track upstream response time, connection counts, failures, and 5xx responses. Compare backend logs so a proxy-level success rate does not hide one overloaded member.

Troubleshooting common symptoms

NGINX starts, but every request returns an upstream error

• Check that the upstream name in proxy_pass exactly matches the declared group.
• Verify the NGINX host can resolve each backend name and reach its port.
• Inspect the error log for connection refusal, timeout, TLS, or name-resolution details.
• Request the backend directly from the NGINX host to separate an application failure from a proxy-routing failure.

Only one server receives traffic

• Confirm the other servers are not marked unavailable after passive failures.
• Look for a weight that heavily favors one member.
• Check whether a hash method or a single long-lived connection makes the traffic appear sticky.
• Ensure your test sends multiple independent requests rather than reusing one connection and measuring only connection assignment.

Users lose login state after a few requests

• Determine whether sessions are stored locally on each backend.
• Move session data to a shared store or implement a supported replication strategy.
• If a short-term affinity workaround is necessary, test ip_hash against NAT, mobile clients, and failover behavior.
• Confirm that cookie, authorization, and forwarding headers survive the proxy path.

A backend is still receiving requests after it fails

• Remember that passive checks require live transactions; no request means no new passive observation.
• Review the failure parameters and proxy timeouts for the upstream.
• For proactive detection, use an appropriate NGINX Plus active check and a meaningful health endpoint.
• Check for existing long-lived connections, which may continue until the protocol or application closes them.

Least connections does not distribute as expected

• Connection count is not the same as CPU use, request latency, or response size.
• At low load, separate worker state can make selection less accurate.
• Use a shared upstream zone when coordinated counters are important, then measure again under representative traffic.
• Inspect whether one backend is handling long-lived streams that legitimately keep its active-connection count high.

Design choices at a glance

Operational practices that prevent avoidable outages

• Keep backend application versions compatible during rolling deployments. A load balancer cannot reconcile incompatible session formats or API responses.
• Make health endpoints reflect the dependencies required for real traffic, but keep them inexpensive enough to check frequently.
• Record which upstream member handled a request so an intermittent error can be traced to one backend.
• Alert on rising upstream latency and 5xx responses, not only on whether the NGINX process is running.
• Test one-member failure regularly in a maintenance window. A configuration that looks correct on paper may fail because of firewall rules, session handling, or an overlooked long-lived connection.
• Keep the NGINX configuration under version control, and retain the last known-good file for rollback after every reload.

The reusable upstream group is the center of the design. Select the method according to workload and state, use passive or active health behavior with its edition limits understood, coordinate workers with a zone when needed, and verify failure, recovery, and session behavior with real requests before declaring the pool ready.

Quick Recap

SaleBestseller No. 1

NGINX Cookbook: Advanced Recipes for High-Performance Load Balancing

$37.53

SaleBestseller No. 2

NGINX Cookbook: Over 70 recipes for real-world configuration, deployment, and performance

$23.69

SaleBestseller No. 3

NGINX Unit Cookbook: Recipes for Using a Versatile Open Source Server

$12.31

Bestseller No. 4

NGINX HTTP Server: Harness the power of NGINX with a series of detailed tutorials and real-life examples

$51.99

Bestseller No. 5

NGINX Cookbook: Advanced Recipes for High-Performance Load Balancing

$26.99

Product prices and availability are accurate as of the date/time indicated and are subject to change. Any price and availability information displayed on Amazon at the time of purchase will apply.