Are failed new connections caused by an exhausted conntrack table?

When application processes are running normally and existing long-lived connections still carry traffic, intermittent timeouts on new requests do not necessarily point to the application's listen queue. On a dedicated server that handles stateful firewalling or network address translation (NAT), the Linux connection tracking table can also become a bottleneck. The key is to correlate the timing of failures with table occupancy and kernel counters—not to raise the limit simply because there are many connections.

This article uses Debian 13 as an example. Run all checks in the host's network namespace. After paying for a CloudUDP server, retrieve its credentials from the console and connect to Linux over SSH. If multiple addresses are assigned, use the first address. The following steps do not modify network rules or clear existing connection states.

1. Confirm that affected traffic passes through connection tracking

Connection tracking (conntrack) records a flow's protocol, addresses, state, and timeout for use by state matching and NAT. An application connection is not the same thing as an application thread, and its tracking entry may remain for some time after the application closes the connection. Rapid creation of short-lived connections, abnormal scanning activity, and forwarded traffic can all increase table occupancy.

First, install the inspection tool and read the configuration:

sudo apt-get update
sudo apt-get install conntrack
sysctl net.netfilter.nf_conntrack_count
sysctl net.netfilter.nf_conntrack_max
sudo conntrack -S
sudo nft list ruleset

Rules containing ct state or NAT are a reason to examine tracking capacity. However, their absence does not prove that no tracking takes place. If a parameter is reported as missing, first check the modules and the actual traffic path. Do not load a module blindly just to make the expected output appear.

Results from inside a container are not a substitute for results from the host. Host forwarding and traffic inside a container may traverse different namespaces, so record where each command was run. Otherwise, you may mistakenly use low occupancy in a container to explain packet drops on the host.

Do not use conntrack -F as a routine recovery measure. Flushing the table removes existing state, can disrupt sessions that depend on state matching or NAT, and destroys evidence that should be preserved.

2. Collect occupancy and error deltas over the same interval

nf_conntrack_count is the current number of entries, while nf_conntrack_max is the capacity limit. Approaching the limit indicates reduced headroom, but does not by itself prove that requests are failing for this reason. conntrack -S reports statistics per processor. Sum each counter across processors and compare the totals from two samples; do not treat a single output line as the total for the server.

umask 077
out=$(mktemp -d /var/tmp/ct-check.XXXXXX)
for sample in 1 2; do
  {
    date -Is
    sysctl net.netfilter.nf_conntrack_count
    sysctl net.netfilter.nf_conntrack_max
    sudo conntrack -S
  } > "$out/sample-$sample.txt"
  if [ "$sample" -eq 1 ]; then
    sleep 10
  fi
done
sudo journalctl -k --since "-10 min" --no-pager \
  > "$out/kernel.txt"
printf '%s\n' "$out"

The sampling interval here is only an example; your samples should cover the period when application failures occur. Focus on whether insert_failed, drop, and early_drop keep increasing, and whether the kernel reports that the connection tracking table is full. Available fields can vary by kernel version, so the absence of a particular field does not establish that the system is healthy.

Also save the application's count of failed new connections, request volume, and original error messages. If the server has just rebooted, the baseline for cumulative counters has changed. Do not subtract pre-reboot values from post-reboot values. Error counters that stop increasing during a quiet period do not prove that capacity is sufficient at peak load.

3. Determine why entries are accumulating

Start by distinguishing normal growth in concurrency from an abnormal connection creation rate. Compare incoming requests, outbound connections initiated by the application, forwarded traffic, and connection reuse. Even short-lived connections can accumulate when their creation rate is high. For long-lived connections, occupancy depends more directly on actual concurrency.

Do not start with a full table dump

On a busy host, listing every entry adds diagnostic overhead, and the output includes communication addresses. Read summary counters first. If you genuinely need entry details, restrict the protocol and output size:

sudo timeout 5s conntrack -L -p tcp -o extended 2>/dev/null \
  | head -n 100

This output is a biased sample, not a statistical breakdown of connection states. A timeout or an early closure of the pipe can also cause a nonzero exit status; that is not evidence of table corruption. Store address records only in a restricted directory, and remove unrelated application information before submitting a support ticket.

At the same time, check listening processes, application errors, and upstream responses. If the table has headroom and the relevant counters are not increasing, investigate application blocking, routing, or other locations where packets may be dropped instead of continuing to tune conntrack.

Do not start by shortening timeouts for every protocol. Timeouts determine how long idle connection state is retained, and reducing them too aggressively can affect long-lived connections. Problems such as failure to reuse pooled connections or uncontrolled retries should be fixed on the caller side.

4. Make a reversible change within the memory budget

Raising the limit merely allows more entries to exist; it does not fix the source of abnormal traffic. Entries consume kernel memory, and their actual overhead depends on factors such as protocol extensions and kernel configuration. A fixed “bytes per entry” figure cannot provide an exact memory budget.

Before making a change, record free -h and /proc/meminfo, and inspect kernel object caches with sudo slabtop -o. Slab is the kernel's mechanism for caching objects. Statistics for connection tracking objects do not account for all associated memory costs.

If the evidence supports a capacity shortage and memory headroom is available, test a small increase. The following raises the original limit by one quarter. This is a trial increment, not a recommended fixed configuration:

old=$(sysctl -n net.netfilter.nf_conntrack_max) || exit 1
case "$old" in
  ''|*[!0-9]*) exit 1 ;;
esac
new=$((old + old / 4))
printf 'old=%s new=%s\n' "$old" "$new"
sudo sysctl -w net.netfilter.nf_conntrack_max="$new"

Keep this session open and retain the original value, then sample again at a comparable request volume. If failures do not improve, do not keep raising the parameter. Restore the original value and examine the other evidence:

sudo sysctl -w net.netfilter.nf_conntrack_max="$old"

Check current occupancy before rolling back. If it already exceeds the original limit, first reduce the pressure from new connections; otherwise, the rollback may trigger another capacity shortage. Do not change hash buckets, protocol timeouts, and firewall rules at the same time. Keep the test limited to one variable.

5. Use application results to decide whether to persist the change

Validation requires three kinds of evidence together: the success rate of new connections recovers at a comparable workload, the relevant error counters stop increasing persistently, and memory and kernel object caches show no new pressure. A lower occupancy percentage alone does not mean the problem is fixed.

Once the change is confirmed effective, write the validated value to /etc/sysctl.d/90-conntrack-capacity.conf. Before doing so, check whether other configuration files define the same parameter, to prevent load order from overriding it. Load only this file, without also applying other unvalidated settings on the server:

sudo sysctl -p /etc/sysctl.d/90-conntrack-capacity.conf
sysctl net.netfilter.nf_conntrack_max

The file should contain one line in the form net.netfilter.nf_conntrack_max = <validated integer>, with the explanatory placeholder replaced by the actual number. After the next reboot scheduled by operations, check the effective value and any loading errors again. Do not immediately reboot a production server just to verify the parameter.

Finally, retain the original value, the new value, the sampling interval, request volume, counter deltas, and rollback conditions. Managing connection tracking capacity is not about reaching a larger number. It is about explaining connection failures, proving that a change works, and checking the same evidence again when the workload changes.

Frequently asked questions