Blog

Benchmarking port scanners honestly

We withdrew our own "2.62x faster than Nmap". Here is the protocol that replaced it: a ground truth, every run checked, and the results, including the ones that do not flatter us.

For a while, tcpcat's README said it was 2.62× faster than Nmap and 4.69× faster than naabu. Both numbers were wrong. They were measured while a bug made tcpcat scan only the first of the two targets it was given, and nothing checked whether each tool had found the open ports. We withdrew them and built a benchmark that would have caught both mistakes. This post describes that protocol and what it found, including the parts that do not flatter us.

The rule: a fast wrong answer is not a result

A port scanner's output is a list of open ports. Timing the run without checking that list measures how quickly a tool gives up, not how quickly it scans. So the benchmark starts from a ground truth:

  • Two target containers on a private Docker network. Each one listens on 40 TCP ports drawn from a fixed seed: 8 well-known ones and 32 anywhere in 1–65535. Every other port is closed and answers with a RST.
  • The seed alone determines the 80 open ports, so the harness knows them before any tool runs.
  • Every tool scans ports 1–65535 on both targets: 131,070 probes per run, from the same scanner container, on the same interface, under the same rate ceiling.
  • Each tool runs five times under hyperfine. Every run is compared with the ground truth. A tool is valid only if every one of its runs found all 80 open ports with no false positive.
  • Speed ratios are only computed between valid tools. If two means differ by less than the two tools' combined run-to-run spread, the report says "on par" instead of printing a ratio.

That last rule came from our own report. It once printed "1.13× faster than Nmap" for two runs whose ranges overlapped.

Giving every tool a fair setting

Each tool gets the rate ceiling and its own sensible defaults, not a configuration that makes it look bad:

  • Nmap gets -T4 --max-rate. Forcing it with --min-rate 100000 made it find 21 of 80 ports in an earlier lab, which measures a misconfigured Nmap, not Nmap.
  • masscan sends every probe to a single next-hop MAC address, the gateway's by default. On a Docker network that would route its probes through the host while the other tools reach the targets directly. So each target gets its own masscan instance, aimed at that target's MAC with --router-mac, running side by side at half the rate. --wait 1 replaces the 10-second default, which is sized for Internet round trips.
  • tcpcat runs both of its engines: raw sockets (-sS) and AF_XDP (--ebpf --ebpf-copy).

Results

One session on November 17, 2026: a KVM server with 8 vCPUs, Linux 6.12, five runs per tool. tcpcat was the v1.7.0 development branch, with the raw-socket fix described below but before the eBPF fixes. The other tools were Nmap 7.93, naabu 2.7.0 and masscan 1.3.2.

Rate ceiling of 1,000,000 packets/s (each tool as fast as it will go)

ToolMean ± σCPU per runOpen ports, worst runValid
tcpcat -sS1.32 s ± 0.126.7 s80 / 80yes
Nmap4.58 s ± 0.344.3 s80 / 80yes
tcpcat --ebpf5.39 s ± 0.199.0 s80 / 80yes
naabu8.62 s ± 0.2040.5 s76 / 80no
masscann/an/a0 / 80no

Rate ceiling of 25,000 packets/s (131,070 probes cannot take less than 5.24 s)

ToolMean ± σCPU per runOpen ports, worst runValid
tcpcat -sS5.44 s ± 0.0211.0 s80 / 80yes
Nmap5.61 s ± 0.034.7 s80 / 80yes
tcpcat --ebpf5.81 s ± 0.199.2 s80 / 80yes
naabu17.76 s ± 0.0549.6 s80 / 80yes
masscann/an/a0 / 80no

What the numbers say

  • At full speed, tcpcat's raw-socket engine is the fastest valid tool: 1.32 s against 4.58 s for Nmap, about 3.5×, with every open port found on every run.
  • Nmap is the most CPU-efficient tool by a wide margin. Its adaptive timing spends about 4.5 s of CPU per run where tcpcat spends 6.7 to 11 s. If your constraint is CPU, not wall time, Nmap is still the better choice.
  • At a shared rate ceiling, the good tools tie at the floor. Speed only separates scanners when the network lets them go fast.
  • naabu misses ports when pushed. It found 76 of 80 (74 in another session) at full speed, and was valid at 25,000 pps, but about 3× slower than the others.
  • masscan 1.3.2 never produced a valid run here. After its scan, its countdown went negative ("waiting -538-secs") and it ignored Ctrl-C. We reproduced the hang with a single instance, with two side by side, and through the gateway, in roughly one run out of two. When it does exit, it finds all 20 ports of a short-range test. The harness now stops it with a watchdog and records why. We don't treat that as a measurement of masscan's speed. It is a measurement of that build in this environment.
  • tcpcat's own eBPF engine was slower than its raw-socket engine. On a virtual interface, AF_XDP runs in copy mode, where it has no advantage and a single-threaded send path. Finding out why also uncovered a bug that made it report unprobed ports as filtered. That story has its own post.

The fix that mattered most was boring

Earlier the same day, on v1.6.0, the raw-socket engine took 4.52 s and 25 s of CPU per run, keeping about five cores busy. A micro-benchmark of a single probe found roughly 25 system calls around its one sendto. Each probe opened its own raw socket, bound it to the interface, and found its source address with a UDP "dial" to 8.8.8.8 plus two netlink dumps. Then it closed the socket. Sharing one send socket per interface and caching the source address took the CPU cost of a probe from about 200 µs to 21–42 µs, and the scan from 4.52 s to 1.32 s. There was no clever packet path behind it: we just stopped doing setup work 131,070 times.

What these numbers are not

They describe one lab and should be quoted with it:

  • The scanner talks to the targets over a virtual interface (veth), not a network card. There is no wire latency and no loss.
  • The server is shared and busy. Its load went from about 1.5 to 6 during runs.
  • On this host, traffic between containers crosses about 770 iptables rules installed by Docker, k3s and flannel. That per-packet cost caps every tool. Nmap finished the same workload in 0.94 s in a laptop Docker lab, against 4.6 s here.
  • Closed ports answer with a RST, so the protocol measures the fast path. It does not exercise retries against filtered ports.

The comparison between tools is meaningful because they share all of this. The absolute numbers are not a promise about your network.

Run it yourself

git clone https://github.com/NycolazSec/tcpcat && cd tcpcat
./benchmarks/run.sh                                        # 25,000 pps ceiling, 3 runs
RATE=1000000 RUNS=5 ./benchmarks/run.sh                    # as fast as each tool goes
TOOLS="tcpcat-ebpf tcpcat-syn nmap naabu masscan" ./benchmarks/run.sh

The report records the kernel, CPU, interface, driver, tool versions, every run's timing and CPU time, packet drops, and, for every run, which open ports it found. The method is described in benchmarks/README.md.

We would especially like results from real network cards, from AF_XDP zero-copy drivers, and from anyone who can get masscan 1.3.2 to exit cleanly in this setup. If our harness is unfair to a tool, open an issue: the point of a ground truth is that anyone can check it.