Production incident: the bridge TCP connection went zombie — outbound sends
kept working for ~10 h while inbound packets stopped entirely, and
/api/status stayed connected:true because nothing detects inbound silence on
an established connection.
- Track lastPacketAt: updated on every chunk received from the bridge in the
lowest-level socket 'data' path (any frame kind, decodable or not) and reset
when a connection successfully starts, so a fresh connection can't
self-trigger.
- Idle watchdog: a 30 s interval checks connected AND now - lastPacketAt >
MESH_IDLE_TIMEOUT_MS (new env, default 600000 = 10 min) — logs WARN with the
idle duration and destroys the socket so the EXISTING connect/retry loop
re-establishes it with a fresh sync. No changes to connect-error retry or
backoff; this only adds detection for established-but-silent connections.
- TCP keepalive: socket.setKeepAlive(true, 30_000) on the bridge socket —
cheap insurance against dead peers at the TCP level.
Tests (fake-bridge based): silence past an overridden timeout forces a
disconnect + reconnect; inbound traffic keeps resetting the idle clock so no
false trigger; the default timeout does not fire in a test window; and
setKeepAlive(true, 30_000) is asserted on the socket. checkIdle() is public so
tests drive a tick without waiting for the real interval (like flushHistory()).
kamilkrzywda