Force reconnect when the bridge connection goes silent (v0.21.1)
CI / build + unit (pull_request) Successful in 1m8s

Production incident: the bridge TCP connection went zombie — outbound sends
kept working for ~10 h while inbound packets stopped entirely, and
/api/status stayed connected:true because nothing detects inbound silence on
an established connection.

- Track lastPacketAt: updated on every chunk received from the bridge in the
  lowest-level socket 'data' path (any frame kind, decodable or not) and reset
  when a connection successfully starts, so a fresh connection can't
  self-trigger.
- Idle watchdog: a 30 s interval checks connected AND now - lastPacketAt >
  MESH_IDLE_TIMEOUT_MS (new env, default 600000 = 10 min) — logs WARN with the
  idle duration and destroys the socket so the EXISTING connect/retry loop
  re-establishes it with a fresh sync. No changes to connect-error retry or
  backoff; this only adds detection for established-but-silent connections.
- TCP keepalive: socket.setKeepAlive(true, 30_000) on the bridge socket —
  cheap insurance against dead peers at the TCP level.

Tests (fake-bridge based): silence past an overridden timeout forces a
disconnect + reconnect; inbound traffic keeps resetting the idle clock so no
false trigger; the default timeout does not fire in a test window; and
setKeepAlive(true, 30_000) is asserted on the socket. checkIdle() is public so
tests drive a tick without waiting for the real interval (like flushHistory()).
This commit is contained in:
2026-09-26 06:50:01 +00:00
parent 73ce49c741
commit af1c77fd54
5 changed files with 145 additions and 2 deletions
+6
View File
@@ -1,5 +1,11 @@
# Changelog
## v0.21.1 — 2026-09-26 06:47
- Bridge zombie-connection fix (production incident: outbound sends kept working ~10 h while inbound packets stopped entirely and `/api/status` stayed `connected:true`). New **idle watchdog**: the service tracks `lastPacketAt`, updated on EVERY chunk received from the bridge in the lowest-level socket `'data'` path (any frame kind, decodable or not) and reset when a connection successfully starts. A 30 s interval checks: if connected AND no inbound traffic for `MESH_IDLE_TIMEOUT_MS` (new env, default **600000** = 10 min), it logs WARN ("no bridge traffic for X ms — forcing reconnect") and destroys the socket — the EXISTING connect/retry loop (`MESH_RECONNECT_MS`) then re-establishes with a fresh sync. No changes to connect-error retry/backoff behavior; this only adds detection for established-but-silent connections.
- TCP keepalive on the bridge socket: `socket.setKeepAlive(true, 30_000)` — cheap insurance against dead peers at the TCP level, complementing the application-level watchdog.
- Tests: 165 → 169 — silence past an overridden timeout forces disconnect + reconnect (fake bridge), inbound traffic keeps resetting the idle clock so no false trigger, default timeout does not fire in a test window, and `setKeepAlive(true, 30_000)` is asserted on the socket.
## v0.21.0 — 2026-09-25 19:59
- `#rss get <name|#tag>` now replies with **3–5 SEPARATE mesh messages** instead of one small summary (owner request: the single-message summary was too small). Each item is its own packet, so no fragmentation — and each rides the trigger's channel threaded as a quote of the command packet.