A deep packet inspection tool for Linux. Version 0.6.0 — complete and in use: a sniffer with flow tracking, protocol classification, reference-data enrichment, TCP stream reassembly, application metadata extraction, file carving and multi-worker scale-out, built around a capture-backend seam that AF_XDP or DPDK can plug into if AF_PACKET ever runs out of headroom.
It links nothing but libc and pthreads. No libpcap, no nDPI, no
libmaxminddb — the classifier, the MaxMind reader and the SHA-256 are all in
this tree, which is why make on a bare toolchain is the whole build.
Capture
- AF_PACKET (TPACKET_V3) mmap-ring capture on a named interface
- L2–L4 decode: Ethernet, 802.1Q/QinQ VLAN, ARP, IPv4, IPv6, TCP, UDP, ICMP/ICMPv6, IPv6 extension-header chains, IPv4/IPv6 fragments
- Live per-packet output, plus a periodic running-stats line
- Rotating
pcapoutput (opens in Wireshark/tcpdump) - Kernel drop accounting (a capture tool that hides drops is lying)
- A
capture_backendseam that AF_XDP / DPDK can plug into
Flows and classification
- Per-worker 5-tuple flow table: direction-symmetric keys, bidirectional counters, TCP teardown tracking, timeout and capacity eviction
- Protocol classification with no external library: 31 payload matchers (HTTP+Host, TLS+SNI, DNS/mDNS/LLMNR+qname, SSH, SMB2, DHCP+hostname, QUIC, WireGuard, MySQL, PostgreSQL, MQTT, STUN, SIP, TFTP+filename and more), backed by a well-known port table
- Every verdict reports its evidence —
payload(we saw the protocol speak) orport(we inferred it) — so a guess never reads as a fact - Rotating JSONL flow records alongside the pcap
- No per-packet allocation: flows and DPI state come from memory pools
Enrichment
- MAC vendor (IEEE OUI), service name (IANA), country/city/coordinates and AS number/organization attached to every flow record
- A from-scratch MaxMind DB (
.mmdb) reader — nolibmaxminddb, no other dependency; the file is mmap'd read-only and shared across workers update-db.sh: fetch, verify, convert, then promote atomically via acurrent ->symlink, keeping the previous version for--rollback- Every source is optional. A missing database means fewer columns, never a failed capture
Reassembly and extraction
- Per-flow, per-direction TCP reassembly: out-of-order segments, retransmits, overlaps, sequence wraparound, and abandoned holes reported rather than hidden
- Application metadata from the reassembled stream, as a stream of events:
- HTTP — method, URL, Host, User-Agent, status, server, content type, response length, elapsed time; one event per transaction, keep-alive and chunked bodies stepped over exactly
- TLS — SNI, ALPN, negotiated version, cipher, and (through TLS 1.2) the certificate subject, issuer and validity, parsed from DER
- DNS — query name and type, response code, and decoded A/AAAA/CNAME/ PTR/MX answers, over both UDP and TCP
- FTP — logins and file transfers with their status; passwords are never recorded
- SMTP — HELO identity, envelope sender and recipients, acceptance
- Bounded by construction: a capped pool of reassembly windows, released the moment a flow has nothing more to say, and flows nothing recognizes are dropped immediately rather than followed
Carving
- Files recovered from cleartext streams into
<PREFIX>.files/, with a"type":"file"record giving the path, size, SHA-256 and sniffed content type - Sources: HTTP response bodies and uploads (Content-Length and chunked), SMTP
message bodies as
.eml, and FTP data connections — correlated to their transfer by tracking thePASV/EPSVannouncement on the control channel - SHA-256 implemented from scratch, so the tool still links nothing but libc
- Off unless you ask for it (
-C), hard-capped per object, in total and by count, and every name from the wire is reduced to one safe component
Scale-out
-w Ncapture threads, each with its own AF_PACKET ring joined to onePACKET_FANOUTgroup and its own flow table, DPI, reassembly and carving — nothing on the packet path is shared, so nothing on it is locked- Both directions of a conversation reach the same worker, which is what makes a per-worker flow table correct; verified on live traffic, not assumed
- Workers pinned to CPUs by default when there is a core each
tune.shfor real: reports and fixes the NIC settings that distort a capture or drop packets before the tool sees them, with--showand--restore--web [ADDR:]PORTserves a live status page — rates and their history, traffic mix, top protocols, per-worker split, and the flows and events as they happen. Off unless asked for, loopback-only unless told otherwise, and compiled into the binary with nothing to install
Application inference
- What was this traffic for?, answered from the names the other stages already recover: the SNI or Host a flow carried, or a name seen resolving to the address it went to, then the AS organization, then the port
- A DNS reverse map learned from the answers this capture parses, so a bare TLS connection to an address with no SNI is still attributed to the name that produced it — the single highest-value signal once everything is encrypted
- About 400 name rules across 23 categories (streaming, messaging, gaming,
updates, ads and telemetry, remote access…), plus AS-organization,
User-Agent, port and protocol tables — all data, in
src/appid/apptable.c - Every verdict carries its evidence, and a recognized application is kept distinct from a name merely reduced to its registrable domain. Inference is never presented as identification
No external dependencies — just a C toolchain.
makeThe pipeline above the capture backend is verified offline, so it needs no interface and no privileges:
make test # decode / dpi / protocol matchers / flow / jsonl / mmdb /
# enrich / reasm / extract (http tls dns ftp smtp) / sha256 /
# carve / appid / web ui / concurrency, plus three fuzz passes
make test-asan # the same suite under AddressSanitizer + UBSan
make test-tsan # the same suite under ThreadSanitizerA clean run reports 388 checks, 0 failures.
The MaxMind reader is checked against a database the test builds itself with a hand-written encoder, so the reader is not merely agreeing with its own author's reading of the format. It was additionally confirmed against real GeoLite2 and DB-IP files. The TLS certificate parser is checked the same way, against a structurally valid DER certificate the test assembles.
The second fuzz pass matters more than the first. Random bytes almost never get past parser dispatch, so they never reach the code that walks a header block, a TLS extension list or a DER structure. That pass starts from traffic each parser accepts and then corrupts it, which is what puts pressure on the length arithmetic where an out-of-bounds read would actually live.
make test-asan is the run that matters: both fuzz passes drive malformed input
through decode, DPI, flow tracking, reassembly and every extractor, which is
where an out-of-bounds read surfaces.
The AF_PACKET backend is the one thing this cannot cover — it needs CAP_NET_RAW
and a real interface. That gap is closed by a separate script, which generates
known traffic on loopback (HTTP, DNS, and a TLS 1.2 handshake with a self-signed
certificate if openssl is present), captures it, and checks that the flows,
the enrichment and the extracted events all came back:
sudo ./scripts/livetest.shIt cleans up after itself and prints a PASS/INCOMPLETE verdict along with the records it recovered.
Packet capture needs privilege. Either run as root, or grant the binary the capabilities once:
make setcap # sudo setcap cap_net_raw,cap_net_admin+eip ./dpitool
./dpitool -i eth0Capture packets and flow records while watching it live:
./dpitool -i eth0 -o /tmp/capture
# -> /tmp/capture.pcap (rotating raw capture)
# -> /tmp/capture.jsonl (flow records and extracted events, one per line)At high rates, drop the per-packet scroll and watch flows and application events instead:
./dpitool -i eth0 -q -F -E --stats-interval 2Recover files from cleartext streams as well:
./dpitool -i eth0 -o /tmp/capture -C
# -> /tmp/capture.files/000001-http-logo.pngOn a fast link, give it a thread per core:
./dpitool -i eth0 -o /tmp/capture -q -w 0 # 0 = one worker per CPUOnly ever capture on networks you're authorized to monitor.
--web starts a small HTTP server alongside the capture. It is off by default;
passing the flag is the only thing that starts it.
./dpitool -i eth0 -q --web 8080 # http://127.0.0.1:8080/
./dpitool -i eth0 -q --web 10.0.0.5:8080 # a specific address
./dpitool -i eth0 -q --web 0 # let the kernel pick the portThe page shows throughput, packet rate and kernel drops with the last few minutes of history for each; the applications the traffic was for and the activity they add up to; the local endpoints doing it and what each one was mostly up to; anything worth a look (cleartext logins, obsolete TLS, file sharing leaving the network); the traffic mix by transport and address family; the protocol breakdown; the per-worker packet split; the pipeline counters; and the flows and extracted events as they happen. It refreshes once a second and holds the last render while a refresh is in flight, so nothing flashes.
It is read-only and has no authentication. A bare port therefore binds
127.0.0.1 and nothing else; to watch a remote capture, forward the port over
ssh rather than exposing it:
ssh -N -L 8080:127.0.0.1:8080 sensor-hostGiving --web an address that is not loopback binds it there and says so at
startup, in as many words. Anyone who can reach that port can read hostnames,
addresses, URIs and everything else the page displays.
Two routes exist, and both are GET:
| Route | What it is |
|---|---|
/ (also /index.html) |
the page — one self-contained HTML file compiled into the binary |
/api/stats |
everything the page draws, as JSON |
/api/stats is a stable enough shape to script against:
curl -s localhost:8080/api/stats | jq '.totals.drops, .history.pps[-1]'The server is one thread, handles one connection at a time, and reads the same
counters the periodic stats line does — it never touches the packet path. With
--web absent it costs nothing at all: no thread, no socket, no branch beyond a
null check per flow.
Protocol classification answers what a flow speaks. On a network where everything is TLS on 443, that answer is correct and nearly useless: a video stream, a chat client and a software update are all "TLS". So there is a second stage that answers a different question — what was this connection for? — from the names the other stages already recover.
It works down a ladder, strongest evidence first:
| Evidence | What it means |
|---|---|
name |
the flow carried the name itself — SNI, HTTP Host, or a DNS question |
user-agent |
the client named itself, in cleartext HTTP |
dns |
this capture saw a name resolve to the address the flow went to |
as-org |
the organization that owns the address |
port |
a well-known port, and nothing better |
protocol |
only the protocol verdict, given a category |
The dns rung is the one that matters most. dpitool already parses DNS answers,
so it keeps a bounded map of address → the name that resolved to it. A later TLS
or QUIC connection carrying no SNI at all still comes back as YouTube, because
something on this network asked for googlevideo.com and got that address. The
map holds 8192 entries, is shared by every worker, and costs one lock per DNS
answer and one per retired flow.
Two honesty rules are built in:
- Inference never masquerades as identification. Every verdict reports the rung it came from, in the JSONL and on the page.
- A recognized application is distinct from a reduced name. If nothing in
the tables matches, the name is cut back to its registrable domain and
reported as that, flagged
"appid_known":false.Netflixis a claim;example-corp.netis just where the packets went.
The tables — 399 domain suffixes across 23 categories, plus 52
AS-organization, 43 User-Agent, 71 port and 36 protocol rules — are plain data
in src/appid/apptable.c. Adding an application is
adding a row. They are a heuristic and they will age; that is why they are
sorted by category and commented rather than generated.
Turn the whole stage off with --no-appid.
# what was this capture actually doing?
jq -r 'select(.type=="flow" and .appid) | [.appid_category, .appid] | @tsv' \
/tmp/capture.jsonl | sort | uniq -c | sort -rn
# only the confident answers
jq -r 'select(.appid_known and .appid_evidence=="name") | .appid' \
/tmp/capture.jsonl | sort -u
# which local host talked to what
jq -r 'select(.type=="flow" and .appid) | [.src_ip, .appid] | @tsv' \
/tmp/capture.jsonl | sort | uniq -c | sort -rn | head-w N runs N capture threads. Each gets its own AF_PACKET ring, joined to a
single PACKET_FANOUT group so the kernel spreads packets across them, and its
own flow table, DPI engine, reassembly windows and carve writer. Nothing on the
packet path is shared between workers, so nothing on it needs a lock.
Two things are shared, and the asymmetry is the point:
| why | |
|---|---|
<PREFIX>-wN.pcap, one per worker |
written per packet — a lock here would sit in the hot path |
<PREFIX>.jsonl, single and locked |
written per flow and per event, rare enough that a mutex costs nothing, and one file is far more useful than N |
Merge the pcaps when you need one file: mergecap -w all.pcap /tmp/capture-w*.pcap.
Both directions of a flow must reach the same worker, or each worker sees
half a conversation and reassembly has nothing to work with. This holds because
the kernel's fanout hash is deliberately direction-independent — the flow
dissector canonicalizes the address and port pair before hashing. That is a
claim about the kernel rather than about this code, so livetest.sh verifies it
against live traffic instead of trusting it: it runs a two-worker capture and
asserts that no flow was recorded with traffic in only one direction.
PACKET_FANOUT_FLAG_ROLLOVER is deliberately not used. It moves packets to a
less busy worker under load, trading exactly the affinity everything above
depends on for slightly less loss.
Before a serious capture, prepare the NIC:
sudo ./scripts/tune.sh --show eth0 # report only, changes nothing
sudo ./scripts/tune.sh eth0 # disable GRO/LRO, grow rings and buffers
sudo ./scripts/tune.sh --restore eth0 # put the offloads backDisabling GRO/LRO is not an optimization — it is the difference between a capture that describes the network and one that describes the driver. With them on, the kernel hands you coalesced frames that never existed on the wire, and TCP reassembly faithfully reconstructs a stream nobody sent.
-C writes objects recovered from HTTP bodies, SMTP messages and FTP data
connections into <PREFIX>.files/, and adds a "type":"file" record to the
JSONL carrying the path, size, SHA-256 and sniffed content type.
It is off by default even when -o is given, which is deliberate:
<prefix>.files/ could reasonably have been just another thing an output prefix
expands into, alongside the pcap and the JSONL. Carving takes bytes chosen by
whoever is on the network and writes them to the local disk under a name partly
derived from what they sent, which is a side effect that should be asked for
rather than inherited from an unrelated flag.
Everything about the implementation follows from that:
- The name on the wire is a hint, never a path. It is reduced to a single safe component from a character whitelist, leading dots are dropped, and the result is always prefixed with a counter — so nothing can escape the directory, collide, or overwrite. A fuzz pass throws 5000 hostile names at it.
- Objects are written under a temporary name and renamed only when complete, so
a partial file is never mistaken for a whole one. Anything truncated or cut
short keeps a
.partialsuffix and is flagged in its record. - Caps are hard: per object, in total, and by count. A capture cannot fill a disk.
<PREFIX>.files/ holds untrusted content by definition. Treat it the way you
would any directory of unknown files.
Enrichment is driven by files fetched with:
./dpitool --update-db # or: ./scripts/update-db.sh| File | Source | Adds |
|---|---|---|
oui.tsv |
IEEE registry | src_mac_vendor, dst_mac_vendor |
services.tsv |
IANA port assignments | service |
GeoLite2-City.mmdb |
MaxMind | country_*, city_*, lat_*, lon_* |
GeoLite2-ASN.mmdb |
MaxMind | asn_*, as_org_* |
The IEEE and IANA files need nothing but network access. GeoLite2 requires a free MaxMind license key:
MAXMIND_LICENSE_KEY=... ./scripts/update-db.sh
./scripts/update-db.sh --no-geo # skip them entirelyEach run installs into a new timestamped directory and flips the current
symlink only once every source has downloaded and parsed into a plausible table,
so an interrupted or rejected update leaves the working set untouched:
./scripts/update-db.sh --status # what is installed
./scripts/update-db.sh --rollback # go back to the previous versionAnything after -- is passed straight through, so the script's flags are
reachable from the binary too:
./dpitool --update-db -- --status
./dpitool --update-db -- --no-geoDatabases live under ${XDG_DATA_HOME:-~/.local/share}/dpitool/db/current.
Override with --db-dir PATH (or $DPITOOL_DB), and turn the whole layer off
with --no-enrich.
A DB-IP "lite" .mmdb works in place of GeoLite2 if you'd rather not register:
the schema keys are the same, so drop it in under the GeoLite2 filename.
-i, --interface NAME interface to capture on (required)
-o, --output PREFIX write <PREFIX>.pcap and <PREFIX>.jsonl (rotating)
-u, --update-db refresh reference databases and exit
(anything after `--` is passed to update-db.sh)
-q, --quiet suppress per-packet lines (stats only)
-F, --flows log each flow as it completes
-E, --events log extracted events (http/tls/dns/ftp/smtp)
-C, --carve write recovered files to <PREFIX>.files/
-w, --workers N capture threads, 0 = one per CPU (default 1)
--web [ADDR:]PORT serve the live status page (default off; a bare
port binds 127.0.0.1, and there is no auth)
--no-pin do not pin workers to CPUs
--no-promisc do not enable promiscuous mode
--no-dpi disable protocol classification
--no-flows disable flow tracking (and JSONL output)
--no-enrich disable reference-database lookups
--no-appid disable application inference
--no-extract disable reassembly and metadata extraction
--carve-max-file N per-object byte cap (default 64 MiB)
--carve-max-total N total byte cap (default 4 GiB)
--carve-max-files N object count cap (default 10000)
--max-reasm N flows reassembled at once (default 2048)
--reasm-window N per-direction reassembly window (default 8192)
--db-dir PATH reference database directory
--frame-size N ring frame size / capture cap (default 2048)
--block-size N ring block size bytes (default 1 MiB)
--block-nr N ring block count, per worker (default 64 -> 64 MiB)
--rotate-bytes N output rotation size, 0=off (default 1 GiB)
--stats-interval S seconds between stats lines (default 5)
--max-flows N flow table capacity, per worker (default 1000000)
--tcp-timeout S idle TCP flow timeout (default 300)
--udp-timeout S idle UDP flow timeout (default 120)
The JSONL file carries two kinds of record, told apart by type. Flow records
("type":"flow") summarize a connection and are written when it is retired
(clean TCP teardown, idle timeout, capacity pressure, or shutdown):
{"type":"flow","start":1700000000.0,"end":1700000005.0,"duration":5.0,
"proto":6,"src_ip":"10.0.0.1","src_port":51000,
"dst_ip":"93.184.216.34","dst_port":443,
"pkts_fwd":10,"bytes_fwd":1500,"pkts_rev":8,"bytes_rev":9000,
"app":"TLS","hostname":"www.example.org","evidence":"payload","confident":true,
"appid":"Netflix","appid_category":"Streaming video","appid_evidence":"dns",
"appid_name":"ichnaea.netflix.com","appid_known":true,
"tcp_flags_fwd":26,"tcp_flags_rev":18,"closed":true,
"service":"https","country_dst":"US","country_name_dst":"United States",
"city_dst":"Los Angeles","lat_dst":34.0544,"lon_dst":-118.2441,
"asn_dst":15133,"as_org_dst":"EDGECAST","src_mac":"00:1b:44:11:22:33",
"src_mac_vendor":"SanDisk Corporation"}src/dst are from the initiator's point of view, and fwd/rev follow that
same direction. Everything from service onward comes from the reference
databases and is present only when a database answered.
app and appid are deliberately different claims. app is the protocol the
bytes spoke, decided by the DPI matchers; appid is what the connection was
probably for, inferred from names — see above.
appid_evidence says which rung of that ladder answered, appid_name is the
name it rests on, and appid_known is false when the "application" is really
just the destination's own domain. The whole group is absent when nothing
matched, and when --no-appid is given.
Event records carry what actually happened on the connection. One flow can produce many, and each is written as it occurs rather than held until the flow ends:
{"type":"http","ts":1700000001.5,"proto":6,"src_ip":"10.0.0.1","src_port":51000,
"dst_ip":"93.184.216.34","dst_port":80,"method":"GET","uri":"/index.html",
"host":"www.example.org","user_agent":"curl/8.5.0","status":200,"reason":"OK",
"content_type":"text/html","server":"nginx","resp_bytes":1256,"elapsed_ms":42}
{"type":"tls","ts":1700000002.1,...,"sni":"www.example.org","alpn":"h2",
"version":"TLS 1.2","cipher":"ECDHE_RSA_AES128_GCM_SHA256",
"cert_issuer":"R3","cert_subject":"www.example.org",
"cert_not_after":"260401120000Z"}
{"type":"dns","ts":1700000000.9,...,"query":"www.example.org.","qtype":"A",
"rcode":"NOERROR","answers":1,"addresses":"93.184.216.34"}
{"type":"file","ts":1700000001.6,...,"source":"http",
"path":"/tmp/capture.files/000001-http-logo.png","name":"logo.png",
"bytes":18244,"sha256":"9f86d081884c7d65...","kind":"png"}Events get the same endpoint enrichment as flow records. A record built from a
stream with a hole in it carries "inexact":true.
Since it's JSONL, it's valid and greppable while still being written:
# every URL fetched
jq -r 'select(.type=="http") | .host + .uri' /tmp/capture.jsonl
# every name looked up, and what it resolved to
jq -r 'select(.type=="dns" and .addresses) | [.query,.addresses] | @tsv' \
/tmp/capture.jsonl | sort -u
# TLS destinations that are not where you expected
jq -r 'select(.type=="tls") | [.sni,.cert_subject] | @tsv' /tmp/capture.jsonl
# flows to another country
jq -r 'select(.type=="flow" and .country_dst!="US")
| [.dst_ip,.country_dst,.as_org_dst] | @tsv' /tmp/capture.jsonl | sort -u- Loopback shows each packet twice. Capturing on
lotaps both the transmit and receive path of the same host, so packets appear duplicated. This is a loopback artifact, not a bug; real interfaces don't double like this. --frame-sizecaps captured length. Frames larger than the ring frame size are truncated (flagged[truncated]). Raise it for jumbo frames.- Tuning for accuracy. Before serious capture, run
sudo ./scripts/tune.sh <iface>to disable receive offloads (GRO/LRO) that would otherwise coalesce segments into frames that never existed on the wire. - Classification is still single-packet; extraction is not. The
appfield on a flow record comes from per-packet matching, so a protocol that only identifies itself across a segment boundary can be missed there. The event records go through reassembly and do see it — ifappsaysUnknownbut anhttpevent exists for the same connection, the event is the better answer. - Check
evidencebefore trustingapp."evidence":"payload"means the protocol was recognized from the bytes on the wire."evidence":"port"means only the port number matched, which is exactly what a service on a non-standard port defeats. The live view marks the weaker case with a trailing?(e.g.MySQL?). - Coverage is deliberate, not exhaustive. The classifier recognizes around
thirty protocols, chosen for analytic value and for having signatures
distinctive enough to match without false positives. Unrecognized traffic is
reported as
Unknownrather than guessed at. Adding a protocol means adding a matcher insrc/dpi/dpi_builtin.cand a case in the test matrix. - MAC vendors describe the local hop. On a routed path the Ethernet source
address belongs to your gateway, not the peer, so
src_mac_vendoridentifies hosts on the capture segment and nothing beyond it. Locally administered addresses (the0x02bit) are left unattributed rather than matched against a registry they were never in. - Geolocation is an estimate. City-level accuracy varies by a lot, VPNs and anycast defeat it outright (the IPv6 resolvers above both geolocate to Montreal), and the coordinates are a database's opinion rather than a measurement. Country and ASN are considerably more dependable than city.
- Reassembly is bounded, and says when it gave up. Each direction gets one
window (8 KB by default) from a capped pool, so at most
--max-reasmflows are reassembled at once — roughly 50 MB at the defaults. A flow that cannot get a window loses extraction, not capture. A hole that later data outruns is abandoned and the stream resynchronizes, with every record from that point marked"inexact":true. The window bounds how much is buffered at once, not how much can pass through it: a segment or an object larger than the window is delivered in pieces, so carved files are not limited by it. What the window does limit is a single message — a header block bigger than it cannot be parsed, and the flow is dropped rather than half-read. - TLS 1.3 hides the certificate. From 1.3 onwards everything after the
ServerHello is encrypted, so
sniis all there is, and the record says"cert":"encrypted"rather than leaving its absence ambiguous. Encrypted Client Hello removes the SNI too, and nothing here can recover it. - Credentials are not recorded. FTP passwords and SMTP AUTH exchanges cross the wire in the clear and this tool can see them. It records that an authentication happened and what it was for, never the secret — a stored capture should not become a credential dump. There is no flag to change this; the raw pcap is still the raw pcap, and that is a deliberate difference.
- Application inference is inference. The name tables are a heuristic and
the DNS map is a correlation, not a proof: a shared CDN address answers for
many names, and the most recent one wins. That is why every verdict carries
its evidence and why
appid_knownexists. Treatnameevidence as strong,dnsas good,as-organdportas hints — and the raw pcap as the record. - The status page is unauthenticated.
--webserves capture metadata — hostnames, addresses, URIs — to anything that can reach the port. That is why a bare port number binds loopback and why binding anywhere else prints a warning. Put it behind an ssh tunnel rather than in front of a network. - Carving is bounded and honest about it. A
.partialsuffix and an"incomplete"or"truncated"flag mean the object on disk is a fragment: the stream ended early, or a cap was reached. Only objects a protocol framed are carved, so their boundaries are known rather than guessed — there is no magic-byte scavenging of unstructured streams.
src/
main.c args, orchestration, signals, capture loop
capture/ backend.h the swap seam (run/stop/get_stats/close)
af_packet.c TPACKET_V3 + PACKET_FANOUT, one ring per worker
decode/ decode.[ch] bounds-checked L2-L4 dissection
flow/ flow.h flow key/record definitions
flowtable.c hash table, eviction, direction accounting
dpi/ dpi.[ch] engine seam + selection
dpi_builtin.c payload matchers + port table
reasm/ tcp_reasm.[ch] per-direction TCP stream reassembly
extract/ extract.[ch] engine, event model, parser dispatch
http.c tls.c dns.c ftp.c smtp.c
carve.[ch] recovered objects on disk, safely named
db/ mmdb.[ch] MaxMind DB reader (mmap, no dependencies)
enrich/ enrich.[ch] OUI / services / geo / ASN lookups
output/ pcap_writer rotating classic-pcap writer
jsonl_writer rotating JSONL flow records
live per-packet terminal line
appid/ appid.[ch] application inference: names, DNS map, evidence
apptable.c the name/ASN/User-Agent/port tables — data, edit freely
web/ webui.[ch] optional status server (--web): one thread, two routes
page.html the status page — edit this one
page.c generated from page.html by scripts/embed-page.sh
util/ stats per-worker counters, summed for the summary
mempool fixed-size object pools
hash FNV-1a + avalanche for flow keys
sha256 content hashes for carved objects
tests/ selftest.c offline verification + fuzz pass
scripts/ tune.sh update-db.sh livetest.sh embed-page.sh
Everything described above is built, tested and in use. There is no half-finished stage: capture, flow tracking, classification, enrichment, reassembly, extraction, carving, application inference, multi-worker scale-out and the status page are all complete, and the offline suite covers every one of them.
Two things were considered and deliberately not built:
- An AF_XDP capture backend. The
capture_backendseam exists and AF_XDP would drop into it, but AF_PACKET with a tuned NIC has not been the bottleneck at the rates this runs at. Adding a second backend means a second thing to keep correct for a gain nothing has yet asked for. The seam is the hedge; taking it is a later decision, made with a profile in hand. - A full-screen TUI. The summarized live view is
--webinstead: a status page keeps the terminal free for the per-packet and per-flow output that is already there, and it works over ssh without a terminal that agrees about escape sequences.
The parts most likely to need attention are not code. The application tables in
src/appid/apptable.c are a heuristic about a moving target and will age; so
will the reference databases behind --update-db. Both are data, and both are
meant to be edited or refreshed without touching anything else.