Compare commits

..
16 Commits
Author SHA1 Message Date
kamilkrzywda 46d32b3553 Merge pull request 'Fix unhandled ECONNRESET in fake-bridge teardown — npm test exits 0 again (v0.21.8)' (#51) from fix/spec-teardown-econnreset into main
CI / build + unit (push) Successful in 1m11s
2026-09-27 09:18:09 +02:00
kamilkrzywda e20e11e2d5 Fix npm test exit 1 — uncaught ECONNRESET from fake-bridge sockets without error handlers (v0.21.8)
CI / build + unit (pull_request) Successful in 46s
2026-09-27 07:13:56 +00:00
kamilkrzywda 0b20df242c Merge pull request 'Fix double ScheduleModule.forRoot() — all cron jobs fired twice (v0.21.7)' (#50) from fix/schedule-double-registration into main
CI / build + unit (push) Failing after 34s
2026-09-27 08:52:35 +02:00
kamilkrzywda a16b6c19d4 Fix double ScheduleModule.forRoot() registration doubling every cron/interval job (v0.21.7)
CI / build + unit (pull_request) Failing after 22s
2026-09-27 06:51:50 +00:00
kamilkrzywda 9e558fd95e Merge pull request 'Add #help command with short list of all commands (v0.21.6)' (#49) from fix/help-command into main
CI / build + unit (push) Failing after 34s
2026-09-26 12:54:06 +02:00
kamilkrzywda 70be8bdaa7 Add #help command — short single-message list of all commands (v0.21.6)
CI / build + unit (pull_request) Failing after 21s
Owner request: 'now add #help with short list of all command'. Registered in
the CommandService constructor next to #ping; returns a SINGLE multi-line
string (one packet, ~203 chars) — not an array, which would drip-feed one
spaced packet per line through the v0.21.5 MESH_MULTI_SEND_DELAY_MS loop.

Also hardened the v0.21.5 multi-send-spacing spec: its wall-clock assertion
('first send < 8 ms after t0') flaked under full-suite load — replaced with
fake timers, which assert deterministically that line 1 is out before any time
elapses and lines 2-3 land exactly one delay apart.
2026-09-26 10:52:18 +00:00
kamilkrzywda 847c3d47d0 Merge pull request 'Space multi-line command replies 5 s apart (v0.21.5)' (#48) from fix/multi-send-spacing into main
CI / build + unit (push) Failing after 46s
2026-09-26 12:31:22 +02:00
kamilkrzywda 1b768b14a7 Space multi-line command replies MESH_MULTI_SEND_DELAY_MS apart (v0.21.5)
CI / build + unit (pull_request) Failing after 21s
Production incident: '#rss get lublin' returned an array of lines and
CommandService sent each one back-to-back within the same second — on the real
mesh 3 of 4 items were dropped in RF (the owner saw only the last one).

CommandService.onRx now waits MESH_MULTI_SEND_DELAY_MS (new env, default 5000 =
5 s, read at construction like the other timing knobs) between consecutive
lines of an array reply — no delay before the first line. The delay sits in the
shared per-line loop, so both the respond() path and the legacy sendText
fallback are covered by one await.
2026-09-26 10:29:21 +00:00
kamilkrzywda 3f6c02d71e Merge pull request 'LLM timeout default 60 s → 180 s (v0.21.4)' (#47) from fix/llm-timeout into main
CI / build + unit (push) Failing after 38s
2026-09-26 11:55:54 +02:00
kamilkrzywda 98b9ec5185 LLM_TIMEOUT_MS default 60 s -> 3 min (v0.21.4)
CI / build + unit (pull_request) Failing after 21s
Production incident: '#rss get lublin' at 9:41 AM failed with 'The operation
was aborted due to timeout' — the LlmService call hit its 60 s cap while the
local OpenAI-compatible server (qwen3.8-27b, uncontrollable reasoning tokens)
was slow/busy. The same endpoint answers a trivial prompt in ~1 s when idle,
so this is latency variance on the server side, not a dead endpoint.

Bump the LLM_TIMEOUT_MS fallback from 60000 to 180000 (3 min headroom for
slow reasoning periods); still env-overridable, only the default changed.
2026-09-26 09:52:18 +00:00
kamilkrzywda 0722c82359 Merge pull request 'Reconnect tuning: 1 min idle timeout + 5 min periodic autoreconnect (v0.21.3)' (#46) from fix/reconnect-tuning into main
CI / build + unit (push) Failing after 42s
2026-09-26 11:37:41 +02:00
kamilkrzywda 1ae8d7a926 Idle timeout default 60 s; lazy periodic autoreconnect every 5 min (v0.21.3)
CI / build + unit (pull_request) Failing after 22s
The v0.21.2 frame-based watchdog works in production (it fired today: 'no
decoded bridge frames for 628474 ms — forcing reconnect', and the reconnect
delivered queued packets), but the bridge/radio side at 192.168.1.4 keeps
killing inbound RX within ~20 min of a fresh sync, repeatedly. Two tuning
changes requested by the owner:

- MESH_IDLE_TIMEOUT_MS default lowered from 600000 (10 min) to 60000 (1 min).
- New lazy periodic autoreconnect: MESH_RECONNECT_INTERVAL_MS (new env,
  default 300000 = 5 min, read at construction like the others). An interval
  started in onModuleInit unconditionally logs and destroys the socket so the
  existing close -> scheduleReconnect loop re-establishes with a fresh sync on
  schedule; cleared in onModuleDestroy. When not connected at tick time,
  destroy is a no-op and the retry loop owns recovery — no special-casing.

Tests: 171 -> 172 — periodic reconnect observed end-to-end with an overridden
short interval (fake bridge disconnect + re-establish); existing watchdog tests
pass unchanged, comments updated to the new 60 s default.
2026-09-26 09:34:30 +00:00
kamilkrzywda 77459adc13 Merge pull request 'Watchdog: track decoded frames, not raw bytes (v0.21.2)' (#45) from fix/watchdog-frame-tracking into main
CI / build + unit (push) Failing after 21s
2026-09-26 10:53:28 +02:00
kamilkrzywda 33a88f3d83 Watchdog tracks decoded frames, not raw bytes; buffer-overflow backstop (v0.21.2)
CI / build + unit (pull_request) Failing after 39s
Second production zombie slipped past v0.21.1's watchdog: text+telemetry RX
dead for >1 h while TX worked and the watchdog never fired. Root cause proven
at runtime via /proc/net/snmp — InSegs delta = 2 segments per 30 s (OutSegs 0):
the bridge peer keeps dribbling small NON-frame bytes (~every 15 s, keepalive
noise or a desynced stream) and v0.21.1 updated lastPacketAt on EVERY socket
chunk, so 'no traffic for MESH_IDLE_TIMEOUT_MS' was never true even though zero
valid Meshtastic frames decoded.

- The 'data' handler no longer updates lastPacketAt per chunk; it is updated
  only when extractFrames yields >=1 valid packet (the reset in the 'connect'
  handler is unchanged), so a stream that decodes nothing trips the watchdog
  after MESH_IDLE_TIMEOUT_MS even while raw bytes keep arriving.
- Buffer-overflow backstop: if the undecoded frame buffer exceeds
  MESH_MAX_FRAME_BUFFER (new env, default 256000) — e.g. one data event of a
  desynced stream that never forms frames — log ERROR and destroy the socket;
  the existing close -> scheduleReconnect loop self-heals it. The undecoded
  buffer is also reset on every (re)connect so leftover garbage from a dead
  session can't re-trigger the backstop or corrupt the first frames of a
  healthy new stream.

Tests: 169 -> 171 — raw non-frame bytes do NOT keep the watchdog alive
(garbage-only stream forces disconnect + reconnect, today's incident shape),
and an undecoded chunk over MESH_MAX_FRAME_BUFFER triggers destroy + a fresh
session that decodes normally (cap env-overridden to 100 since Node delivers
socket data in ~64 KB events, far below the 256_000 default).
2026-09-26 08:50:27 +00:00
kamilkrzywda 306756c63b Merge pull request 'Bridge idle watchdog + forced reconnect (v0.21.1)' (#44) from fix/bridge-idle-reconnect into main
CI / build + unit (push) Successful in 45s
2026-09-26 08:53:06 +02:00
kamilkrzywda af1c77fd54 Force reconnect when the bridge connection goes silent (v0.21.1)
CI / build + unit (pull_request) Successful in 1m8s
Production incident: the bridge TCP connection went zombie — outbound sends
kept working for ~10 h while inbound packets stopped entirely, and
/api/status stayed connected:true because nothing detects inbound silence on
an established connection.

- Track lastPacketAt: updated on every chunk received from the bridge in the
  lowest-level socket 'data' path (any frame kind, decodable or not) and reset
  when a connection successfully starts, so a fresh connection can't
  self-trigger.
- Idle watchdog: a 30 s interval checks connected AND now - lastPacketAt >
  MESH_IDLE_TIMEOUT_MS (new env, default 600000 = 10 min) — logs WARN with the
  idle duration and destroys the socket so the EXISTING connect/retry loop
  re-establishes it with a fresh sync. No changes to connect-error retry or
  backoff; this only adds detection for established-but-silent connections.
- TCP keepalive: socket.setKeepAlive(true, 30_000) on the bridge socket —
  cheap insurance against dead peers at the TCP level.

Tests (fake-bridge based): silence past an overridden timeout forces a
disconnect + reconnect; inbound traffic keeps resetting the idle clock so no
false trigger; the default timeout does not fire in a test window; and
setKeepAlive(true, 30_000) is asserted on the socket. checkIdle() is public so
tests drive a tick without waiting for the real interval (like flushHistory()).
2026-09-26 06:50:01 +00:00
11 changed files with 402 additions and 11 deletions
+3 -3
View File
@@ -12,7 +12,7 @@
## Verification ## Verification
1. **Local checks** — `npm run build` (nest build) and `npm test` (vitest; no e2e suite yet). Both must pass before a PR is opened. 1. **Local checks** — `npm run build` (nest build) and `npm test` (vitest; no e2e suite yet). Both must pass before a PR is opened.
2. **Docker** — `docker compose build`, then after `docker compose up -d`: `curl -s -H "Host: bot.dev.kkhost.pl" http://localhost/api/status` returns JSON (`{ connected, myNodeNum, host, port }`). `"connected":false` is normal while the bridge/device is down — the service retries every 5 s. 2. **Docker** — `docker compose build`, then after `docker compose up -d`: `curl -s -H "Host: bot.dev.kkhost.pl" http://localhost/api/status` returns JSON (`{ connected, myNodeNum, host, port }`). `"connected":false` is normal while the bridge/device is down — the service retries every 5 s. An idle watchdog also force-reconnects an ESTABLISHED-but-silent connection (zombie bridge: outbound works, inbound dead) when no DECODED FRAME arrives from the bridge for `MESH_IDLE_TIMEOUT_MS` ms (default 60000 = 1 min; checked every 30 s — raw non-frame bytes do NOT count), an undecoded frame buffer over `MESH_MAX_FRAME_BUFFER` bytes (default 256000) forces a reconnect as a stream-desync backstop, and the socket has TCP keepalive enabled (30 s). A lazy periodic autoreconnect additionally unconditionally re-establishes every `MESH_RECONNECT_INTERVAL_MS` ms (default 300000 = 5 min) for a fresh sync.
## API ## API
@@ -27,5 +27,5 @@
- MESHTASTIC.md — rules for Meshtastic communication (owner authority, language, response protocol); read before acting on mesh messages. - MESHTASTIC.md — rules for Meshtastic communication (owner authority, language, response protocol); read before acting on mesh messages.
- Hourly cron weather report job (`src/weather`): fetches Open-Meteo current conditions and broadcasts a one-line report to the mesh on the pinned send channel; env `WEATHER_CRON`/`WEATHER_LAT`/`WEATHER_LON`/`WEATHER_CITY`. - Hourly cron weather report job (`src/weather`): fetches Open-Meteo current conditions and broadcasts a one-line report to the mesh on the pinned send channel; env `WEATHER_CRON`/`WEATHER_LAT`/`WEATHER_LON`/`WEATHER_CITY`.
- Channel-2 auto-responder (`src/commands`): event-driven — `CommandService.onRx()` is called by `MeshService` for every decoded rx text (right after the notify gate) and dispatches exact, case-sensitive `#command` matches on `COMMAND_CHANNEL` (env, default 2) through a single-registration-point registry (handlers registered in the constructor via `register(name, fn)`; handler gets `{ text, from?, id? }`, returns a reply string — or an array of lines, sent as one message per line through the same routing — or void = no reply). First command: `#ping` → replies `#pong!`. Replies go through `MeshService.respond(id, reply)`: auto-routed (rx → DM back to the original sender unless the original was a broadcast; tx → its destination), riding the original record's channel with `replyId = id` so the recipient app renders it as a threaded quote — `COMMAND_CHANNEL` is only the trigger/listen gate. Only when the trigger has no packet id does it fall back to legacy sendText routing on `COMMAND_CHANNEL`. A failing handler/respond only logs a warn. DI: provided by `CommandModule`'s factory (injects `MeshService`, attaches itself via `attachCommands()`) — no constructor-level cycle; `forwardRef` was tried and left one side null in NestJS 12's injector. Prefix dispatch (`registerPrefix(name, fn)`) lets one handler own a whole family: the `#rss` subcommands live in `src/rss` (`RssService.handle`, registered as prefix `'rss'` from the CommandModule factory — exact matches still win over prefixes). Subcommands: `#rss help`; `#rss add <url> <name>` (validates URL, fetches with 15 s timeout, requires ≥1 titled item, rejects duplicate names case-insensitively; LLM generates 2-4 lowercase tags from up to 8 latest titles — on LLM failure the source is added with empty tags and the reply says so); `#rss remove <name>`; `#rss tags` (all distinct tags); `#rss sources`; `#rss get <name|#tag>` (LLM headline summary as 3-5 SEPARATE mesh messages: prompt asks for exactly 3-5 lines, one item per line, `<short header>: <one or two short sentences>` with ≤20-char header, no intro/outro/numbering; reply parsed into items — drop empty/separator-less lines, cap at 5, fewer than 3 → send what we got (no retry), zero → no message; each item sent via its own respond() threaded to the trigger packet id, per-line backstop `fitOneLine()` hard-cuts >185-char lines at a word boundary near 180 so every item fits one packet; maxTokens 2500 is headroom because the LLM server's reasoning tokens count against it — see rss.service.ts comment; `#tag` collects items across ALL sources carrying that tag — one dead feed does not sink the summary). Storage: `rss_sources(name TEXT PRIMARY KEY, url, tags, created_at)` table in the EXISTING MESH_DB_PATH file via a second node:sqlite connection (names matched case-insensitively); feed parsing via `fast-xml-parser` (RSS 2.0 + Atom; malformed/no-items → "not a valid feed"); LLM via injected `LlmService.chat()` with explicit maxTokens (50 tags / 2500 summary headroom). - Channel-2 auto-responder (`src/commands`): event-driven — `CommandService.onRx()` is called by `MeshService` for every decoded rx text (right after the notify gate) and dispatches exact, case-sensitive `#command` matches on `COMMAND_CHANNEL` (env, default 2) through a single-registration-point registry (handlers registered in the constructor via `register(name, fn)`; handler gets `{ text, from?, id? }`, returns a reply string — or an array of lines, sent as one message per line through the same routing, spaced `MESH_MULTI_SEND_DELAY_MS` ms apart (default 5000) so each packet has time to land in RF — or void = no reply). First commands: `#ping` → replies `#pong!`; `#help` → short single-message list of all available commands. Replies go through `MeshService.respond(id, reply)`: auto-routed (rx → DM back to the original sender unless the original was a broadcast; tx → its destination), riding the original record's channel with `replyId = id` so the recipient app renders it as a threaded quote — `COMMAND_CHANNEL` is only the trigger/listen gate. Only when the trigger has no packet id does it fall back to legacy sendText routing on `COMMAND_CHANNEL`. A failing handler/respond only logs a warn. DI: provided by `CommandModule`'s factory (injects `MeshService`, attaches itself via `attachCommands()`) — no constructor-level cycle; `forwardRef` was tried and left one side null in NestJS 12's injector. Prefix dispatch (`registerPrefix(name, fn)`) lets one handler own a whole family: the `#rss` subcommands live in `src/rss` (`RssService.handle`, registered as prefix `'rss'` from the CommandModule factory — exact matches still win over prefixes). Subcommands: `#rss help`; `#rss add <url> <name>` (validates URL, fetches with 15 s timeout, requires ≥1 titled item, rejects duplicate names case-insensitively; LLM generates 2-4 lowercase tags from up to 8 latest titles — on LLM failure the source is added with empty tags and the reply says so); `#rss remove <name>`; `#rss tags` (all distinct tags); `#rss sources`; `#rss get <name|#tag>` (LLM headline summary as 3-5 SEPARATE mesh messages: prompt asks for exactly 3-5 lines, one item per line, `<short header>: <one or two short sentences>` with ≤20-char header, no intro/outro/numbering; reply parsed into items — drop empty/separator-less lines, cap at 5, fewer than 3 → send what we got (no retry), zero → no message; each item sent via its own respond() threaded to the trigger packet id, per-line backstop `fitOneLine()` hard-cuts >185-char lines at a word boundary near 180 so every item fits one packet; maxTokens 2500 is headroom because the LLM server's reasoning tokens count against it — see rss.service.ts comment; `#tag` collects items across ALL sources carrying that tag — one dead feed does not sink the summary). Storage: `rss_sources(name TEXT PRIMARY KEY, url, tags, created_at)` table in the EXISTING MESH_DB_PATH file via a second node:sqlite connection (names matched case-insensitively); feed parsing via `fast-xml-parser` (RSS 2.0 + Atom; malformed/no-items → "not a valid feed"); LLM via injected `LlmService.chat()` with explicit maxTokens (50 tags / 2500 summary headroom).
- LLM client (`src/llm`): `LlmService` wraps the local OpenAI-compatible server (default `http://192.168.1.70:1111/v1`, no auth) with native fetch — plain feature module, no Mesh/Schedule dependency. Methods: `listModels()` → `data[].id`; `chat(messages, { maxTokens?, temperature? })` → `choices[0].message.content` (non-2xx throws with status + body snippet); `summarize(text)` = chat() with system prompt "Summarize the following text concisely." Env: `LLM_API_BASE`, `LLM_MODEL` (default `qwen3.8-27b`), `LLM_TIMEOUT_MS` (default 60000, via `AbortSignal.timeout`). - LLM client (`src/llm`): `LlmService` wraps the local OpenAI-compatible server (default `http://192.168.1.70:1111/v1`, no auth) with native fetch — plain feature module, no Mesh/Schedule dependency. Methods: `listModels()` → `data[].id`; `chat(messages, { maxTokens?, temperature? })` → `choices[0].message.content` (non-2xx throws with status + body snippet); `summarize(text)` = chat() with system prompt "Summarize the following text concisely." Env: `LLM_API_BASE`, `LLM_MODEL` (default `qwen3.8-27b`), `LLM_TIMEOUT_MS` (default 180000, via `AbortSignal.timeout`).
+41
View File
@@ -1,5 +1,46 @@
# Changelog # Changelog
## v0.21.8 — 2026-09-27 07:13
- Fixed `npm test` exiting 1 with "Errors 2" even though all tests pass (every CI run red since v0.21.2): the two watchdog specs that force a reconnect destroy the service's socket while data is still pending in its receive buffer (the overflow spec floods 300 KB and destroys on the first ~64 KB chunk; the garbage spec destroys right after its last write), so the kernel sends RST instead of FIN and the fake bridge's accepted server-side sockets emit `read ECONNRESET`. Those test-owned sockets had no `'error'` handler, so Node threw it as an uncaught exception that vitest counts as a run-level failure. `startBridge()` now attaches a no-op `'error'` handler to every accepted socket (single registration point, covers all four spec files using the fake bridge). Test-only change — `fake-bridge.ts` is imported by specs only; no runtime diff, no deploy needed.
## v0.21.7 — 2026-09-27 06:50
- Fixed double registration of scheduled jobs: `ScheduleModule.forRoot()` was called from both `WeatherModule` and `BatteryAlertModule`, so Nest created two ScheduleExplorers, each scanning all providers — every @Cron/@Interval job (hourly weather broadcast, per-minute battery check) fired twice. Removed the call from `battery.module.ts`; the single registration in `weather.module.ts` remains.
- Tests: 174 → 175 — new regression spec (`src/schedule-singleton.spec.ts`) asserts `ScheduleModule.forRoot(` appears exactly once across all `src/**/*.module.ts`.
## v0.21.6 — 2026-09-26 10:37
- New `#help` command (owner request): replies with a short list of all available commands as ONE multi-line message (single packet, not one spaced packet per line) — `#ping`, the `#rss help/add/remove/get/sources/tags` family. Registered in the CommandService constructor next to `#ping`.
- Tests: 173 → 174 — `#help` dispatches through respond() exactly once with text containing both `#ping` and `#rss`. (Also hardened the v0.21.5 multi-send-spacing spec, which flaked under full-suite load: it now uses fake timers to assert deterministically that line 1 goes out before any time elapses and lines 2-3 land exactly one delay apart.)
## v0.21.5 — 2026-09-26 10:26
- Multi-line command replies are now spaced apart: `CommandService.onRx` waits `MESH_MULTI_SEND_DELAY_MS` (new env, default **5000** = 5 s, read at construction) between consecutive lines of an array reply — no delay before the first line. Production incident: `#rss get lublin` sent its items back-to-back within the same second and 3 of 4 were dropped in RF (the owner saw only the last one); each packet now gets time to land.
- Tests: 172 → 173 — a 3-line reply with an overridden short delay asserts all lines are sent, the first goes out immediately (no pre-delay), and each subsequent line lands ≥ the configured gap after the previous one; existing specs pin `MESH_MULTI_SEND_DELAY_MS=0` in `beforeEach` so they stay fast.
## v0.21.4 — 2026-09-26 09:51
- `LLM_TIMEOUT_MS` default raised from 60000 (60 s) to **180000** (3 min): production incident — `#rss get lublin` at 9:41 AM failed with "The operation was aborted due to timeout" because the local OpenAI-compatible server (qwen3.8-27b, uncontrollable reasoning tokens) was slow/busy on that call; the same endpoint answers a trivial prompt in ~1 s when idle, so this is latency variance, not a dead endpoint. Still env-overridable (`LLM_TIMEOUT_MS`), only the fallback changed.
## v0.21.3 — 2026-09-26 09:31
- Idle timeout default lowered from 10 min to **1 min** (`MESH_IDLE_TIMEOUT_MS` fallback `600000` → `60000`) — the v0.21.2 frame-based watchdog works in production (it fired today and the reconnect delivered queued packets), but the bridge/radio side at 192.168.1.4 keeps killing inbound RX within ~20 min of a fresh sync, so a shorter detection window is wanted.
- New **lazy periodic autoreconnect**: `MESH_RECONNECT_INTERVAL_MS` (new env, default **300000** = 5 min, read at construction) — an interval that unconditionally logs ("periodic reconnect — MESH_RECONNECT_INTERVAL_MS elapsed") and destroys the socket so the existing close→scheduleReconnect loop re-establishes with a fresh sync on schedule. When not connected at tick time, destroy is a no-op and the retry loop owns recovery (no special-casing). Started in `onModuleInit`, cleared in `onModuleDestroy`.
- Tests: 171 → 172 — periodic reconnect observed end-to-end with an overridden short interval (fake bridge disconnect + re-establish); existing watchdog tests pass unchanged, comments updated to the new 60 s default.
## v0.21.2 — 2026-09-26 08:47
- Idle watchdog now tracks DECODED FRAMES, not raw bytes (fixes a second production zombie that slipped past v0.21.1): text+telemetry RX was dead for >1 h while TX worked and the watchdog never fired — /proc/net/snmp showed InSegs delta = 2 segments per 30 s, i.e. the bridge peer keeps dribbling small NON-frame bytes (~every 15 s) that kept resetting `lastPacketAt` on every socket chunk even though zero valid Meshtastic frames decoded. The `'data'` handler no longer updates `lastPacketAt` per chunk; it is updated only when `extractFrames` yields ≥1 valid packet (the reset in the `'connect'` handler is unchanged), so a stream that decodes nothing trips the watchdog after `MESH_IDLE_TIMEOUT_MS` even while raw bytes keep arriving.
- Buffer-overflow backstop: if the undecoded frame buffer exceeds `MESH_MAX_FRAME_BUFFER` (new env, default **256000** = 250 KB) the service logs ERROR ("frame buffer overflow — stream desync, forcing reconnect") and destroys the socket → existing close→scheduleReconnect self-heals a desynced stream. The undecoded buffer is also reset on every (re)connect so leftover garbage from a dead session can't re-trigger the backstop or corrupt the first frames of a healthy new stream.
- Tests: 169 → 171 — raw non-frame bytes do NOT keep the watchdog alive (garbage-only stream forces disconnect + reconnect, exactly today's incident shape), and an undecoded chunk over `MESH_MAX_FRAME_BUFFER` triggers destroy + a fresh session that decodes normally. The existing valid-frames-reset-the-clock test now exercises frame-based tracking unchanged.
## v0.21.1 — 2026-09-26 06:47
- Bridge zombie-connection fix (production incident: outbound sends kept working ~10 h while inbound packets stopped entirely and `/api/status` stayed `connected:true`). New **idle watchdog**: the service tracks `lastPacketAt`, updated on EVERY chunk received from the bridge in the lowest-level socket `'data'` path (any frame kind, decodable or not) and reset when a connection successfully starts. A 30 s interval checks: if connected AND no inbound traffic for `MESH_IDLE_TIMEOUT_MS` (new env, default **600000** = 10 min), it logs WARN ("no bridge traffic for X ms — forcing reconnect") and destroys the socket — the EXISTING connect/retry loop (`MESH_RECONNECT_MS`) then re-establishes with a fresh sync. No changes to connect-error retry/backoff behavior; this only adds detection for established-but-silent connections.
- TCP keepalive on the bridge socket: `socket.setKeepAlive(true, 30_000)` — cheap insurance against dead peers at the TCP level, complementing the application-level watchdog.
- Tests: 165 → 169 — silence past an overridden timeout forces disconnect + reconnect (fake bridge), inbound traffic keeps resetting the idle clock so no false trigger, default timeout does not fire in a test window, and `setKeepAlive(true, 30_000)` is asserted on the socket.
## v0.21.0 — 2026-09-25 19:59 ## v0.21.0 — 2026-09-25 19:59
- `#rss get <name|#tag>` now replies with **3–5 SEPARATE mesh messages** instead of one small summary (owner request: the single-message summary was too small). Each item is its own packet, so no fragmentation — and each rides the trigger's channel threaded as a quote of the command packet. - `#rss get <name|#tag>` now replies with **3–5 SEPARATE mesh messages** instead of one small summary (owner request: the single-message summary was too small). Each item is its own packet, so no fragmentation — and each rides the trigger's channel threaded as a quote of the command packet.
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "mesh-bot", "name": "mesh-bot",
"version": "0.21.0", "version": "0.21.8",
"description": "", "description": "",
"author": "", "author": "",
"private": true, "private": true,
+1 -2
View File
@@ -1,10 +1,9 @@
import { Module } from '@nestjs/common'; import { Module } from '@nestjs/common';
import { ScheduleModule } from '@nestjs/schedule';
import { MeshModule } from '../mesh/mesh.module.js'; import { MeshModule } from '../mesh/mesh.module.js';
import { BatteryAlertService } from './battery.service.js'; import { BatteryAlertService } from './battery.service.js';
@Module({ @Module({
imports: [ScheduleModule.forRoot(), MeshModule], imports: [MeshModule],
providers: [BatteryAlertService], providers: [BatteryAlertService],
exports: [BatteryAlertService], exports: [BatteryAlertService],
}) })
+41
View File
@@ -30,6 +30,7 @@ function makeService(
beforeEach(() => { beforeEach(() => {
delete process.env.COMMAND_CHANNEL; // default channel 2 under test delete process.env.COMMAND_CHANNEL; // default channel 2 under test
process.env.MESH_MULTI_SEND_DELAY_MS = '0'; // no spacing under test unless a test opts in — the real default (5000) would slow the suite down
}); });
describe('CommandService.onRx', () => { describe('CommandService.onRx', () => {
@@ -40,6 +41,15 @@ describe('CommandService.onRx', () => {
expect(mesh.sendText).not.toHaveBeenCalled(); expect(mesh.sendText).not.toHaveBeenCalled();
}); });
it('replies #help with a short single-message list of all commands', async () => {
const [svc, mesh] = makeService();
await svc.onRx(2, '#help', 42, 7);
expect(mesh.respond).toHaveBeenCalledTimes(1); // one packet — not one per line
const text = mesh.respond.mock.calls[0][1];
expect(text).toContain('#ping');
expect(text).toContain('#rss');
});
it('falls back to a directed sendText on COMMAND_CHANNEL when the trigger has no packet id (DM)', async () => { it('falls back to a directed sendText on COMMAND_CHANNEL when the trigger has no packet id (DM)', async () => {
const [svc, mesh] = makeService(); const [svc, mesh] = makeService();
await svc.onRx(2, '#ping', 42); // from present but no id — defensive path await svc.onRx(2, '#ping', 42); // from present but no id — defensive path
@@ -124,6 +134,37 @@ describe('CommandService.onRx', () => {
expect(mesh.sendText).not.toHaveBeenCalled(); expect(mesh.sendText).not.toHaveBeenCalled();
}); });
it('spaces consecutive multi-line sends by MESH_MULTI_SEND_DELAY_MS — no delay before the first line', async () => {
process.env.MESH_MULTI_SEND_DELAY_MS = '50'; // short so the test stays fast
let sends = 0;
const respond = vi.fn(async () => {
sends++;
return { id: 1 };
});
vi.useFakeTimers();
try {
const [svc, mesh] = makeService(undefined, respond);
svc.register('multi3', () => ['a: one', 'b: two', 'c: three']);
const p = svc.onRx(2, '#multi3', 42, 9);
// flush microtasks without advancing time — the first line must already be out (no delay before it)
await vi.advanceTimersByTimeAsync(0);
expect(sends).toBe(1);
// each subsequent line waits exactly MESH_MULTI_SEND_DELAY_MS after the previous one landed
await vi.advanceTimersByTimeAsync(50);
expect(sends).toBe(2);
await vi.advanceTimersByTimeAsync(50);
expect(sends).toBe(3);
await p;
expect(mesh.respond).toHaveBeenCalledTimes(3);
} finally {
vi.useRealTimers();
delete process.env.MESH_MULTI_SEND_DELAY_MS;
}
});
it('sends each line of an array reply via the legacy sendText fallback when there is no packet id', async () => { it('sends each line of an array reply via the legacy sendText fallback when there is no packet id', async () => {
const [svc, mesh] = makeService(); const [svc, mesh] = makeService();
svc.register('multi', () => ['one: first item', 'two: second item']); svc.register('multi', () => ['one: first item', 'two: second item']);
+25 -2
View File
@@ -27,6 +27,11 @@ export class CommandService {
process.env.COMMAND_CHANNEL ?? '2', process.env.COMMAND_CHANNEL ?? '2',
10, 10,
); );
// spacing between consecutive multi-line reply messages — back-to-back packets get dropped in RF on a real mesh; each packet needs time to land. Read at construction like the other env config.
private readonly multiSendDelayMs = parseInt(
process.env.MESH_MULTI_SEND_DELAY_MS ?? '5000',
10,
);
// single registration point: the constructor registers every built-in command — add the next one there (or via register()) // single registration point: the constructor registers every built-in command — add the next one there (or via register())
private readonly commands = new Map<string, CommandHandler>(); private readonly commands = new Map<string, CommandHandler>();
// prefix handlers own a whole family (`#rss`, `#rss add ...`) instead of one exact name // prefix handlers own a whole family (`#rss`, `#rss add ...`) instead of one exact name
@@ -38,6 +43,17 @@ export class CommandService {
rss?: RssService, rss?: RssService,
) { ) {
this.register('ping', () => '#pong!'); this.register('ping', () => '#pong!');
// single multi-line string = ONE packet — not an array, which would drip-feed one spaced packet per line
this.register('help', () =>
[
'#ping - alive check',
'#rss help - rss subcommands',
'#rss add <url> <name> - add feed with AI tags',
'#rss remove <name> - drop feed',
'#rss get <name|#tag> - news summary',
'#rss sources / #rss tags - list feeds/tags',
].join('\n'),
);
if (rss) this.registerPrefix('rss', (ctx) => rss.handle(ctx)); if (rss) this.registerPrefix('rss', (ctx) => rss.handle(ctx));
} }
@@ -83,8 +99,15 @@ export class CommandService {
try { try {
const reply = await handler({ text: trimmed, from, id }); const reply = await handler({ text: trimmed, from, id });
if (reply === undefined || reply === null) return; // void = no reply if (reply === undefined || reply === null) return; // void = no reply
// an array reply sends one message per line, all threaded to the same trigger packet id // an array reply sends one message per line, all threaded to the same trigger packet id — spaced so each packet has time to land in RF (no delay before the first line)
for (const line of Array.isArray(reply) ? reply : [reply]) { const lines = Array.isArray(reply) ? reply : [reply];
for (let i = 0; i < lines.length; i++) {
if (i > 0 && this.multiSendDelayMs > 0) {
await new Promise((resolve) =>
setTimeout(resolve, this.multiSendDelayMs),
);
}
const line = lines[i];
if (id !== undefined && this.mesh) { if (id !== undefined && this.mesh) {
// respond() auto-routes each line: rides the original record's channel with replyId = id (threaded quote) // respond() auto-routes each line: rides the original record's channel with replyId = id (threaded quote)
await this.mesh.respond(id, line); await this.mesh.respond(id, line);
+2 -2
View File
@@ -12,9 +12,9 @@ export class LlmService {
private readonly apiBase = private readonly apiBase =
process.env.LLM_API_BASE ?? 'http://192.168.1.70:1111/v1'; process.env.LLM_API_BASE ?? 'http://192.168.1.70:1111/v1';
private readonly model = process.env.LLM_MODEL ?? 'qwen3.8-27b'; private readonly model = process.env.LLM_MODEL ?? 'qwen3.8-27b';
// local 27B model can be slow — generous default // local 27B model can be slow (uncontrollable reasoning tokens) — generous default: 3 min headroom for slow reasoning periods
private readonly timeoutMs = parseInt( private readonly timeoutMs = parseInt(
process.env.LLM_TIMEOUT_MS ?? '60000', process.env.LLM_TIMEOUT_MS ?? '180000',
10, 10,
); );
+2
View File
@@ -10,6 +10,8 @@ export function startBridge(
): Promise<{ port: number; sockets: net.Socket[]; close: () => void }> { ): Promise<{ port: number; sockets: net.Socket[]; close: () => void }> {
const sockets: net.Socket[] = []; const sockets: net.Socket[] = [];
const server = net.createServer((socket) => { const server = net.createServer((socket) => {
// Test-owned socket: the service destroys its end mid-test with unread data pending (overflow/garbage tests), which makes the kernel send RST and this side emit ECONNRESET. Expected teardown noise — without a handler Node throws it as an uncaught exception and vitest fails the whole run.
socket.on('error', () => {});
onSocket?.(socket); // attached at accept time, before any data can arrive onSocket?.(socket); // attached at accept time, before any data can arrive
sockets.push(socket); sockets.push(socket);
}); });
+190
View File
@@ -1,6 +1,7 @@
import { create, fromBinary, toBinary } from '@bufbuild/protobuf'; import { create, fromBinary, toBinary } from '@bufbuild/protobuf';
import * as P from '@meshtastic/protobufs'; import * as P from '@meshtastic/protobufs';
import { BadRequestException, NotFoundException } from '@nestjs/common'; import { BadRequestException, NotFoundException } from '@nestjs/common';
import * as net from 'node:net';
import { DatabaseSync } from 'node:sqlite'; import { DatabaseSync } from 'node:sqlite';
import { beforeEach, describe, expect, it, vi } from 'vitest'; import { beforeEach, describe, expect, it, vi } from 'vitest';
import { BROADCAST_ADDR, extractFrames, frame } from './codec.js'; import { BROADCAST_ADDR, extractFrames, frame } from './codec.js';
@@ -249,6 +250,195 @@ describe('MeshService connect', () => {
}); });
}); });
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
describe('MeshService idle watchdog', () => {
it('forces a reconnect when an established connection goes silent past MESH_IDLE_TIMEOUT_MS', async () => {
const bridge = await startBridge();
process.env.MESH_HOST = '127.0.0.1';
process.env.MESH_PORT = String(bridge.port);
process.env.MESH_RECONNECT_MS = '50'; // fast retry so the re-establishment is observable
process.env.MESH_IDLE_TIMEOUT_MS = '100'; // override — the default (60000) would never fire in a test
const svc = new MeshService();
try {
svc.onModuleInit();
await waitFor(
() => bridge.sockets.length === 1 && svc.status().connected,
);
// zombie case: connected but no inbound traffic at all — wait past the timeout, then drive a watchdog tick
await sleep(250);
svc.checkIdle();
// socket torn down → existing connect/retry loop re-establishes with a fresh session
await waitFor(() => bridge.sockets.length >= 2 && svc.status().connected);
} finally {
delete process.env.MESH_RECONNECT_MS;
delete process.env.MESH_IDLE_TIMEOUT_MS;
svc.onModuleDestroy();
bridge.close();
}
});
it('raw non-frame bytes do NOT keep the watchdog alive — a garbage-only stream forces a reconnect', async () => {
const bridge = await startBridge();
process.env.MESH_HOST = '127.0.0.1';
process.env.MESH_PORT = String(bridge.port);
process.env.MESH_RECONNECT_MS = '50'; // fast retry so the re-establishment is observable
process.env.MESH_IDLE_TIMEOUT_MS = '100'; // override — the default (60000) would never fire in a test
const svc = new MeshService();
try {
svc.onModuleInit();
await waitFor(
() => bridge.sockets.length === 1 && svc.status().connected,
);
// today's incident: the peer keeps dribbling small non-frame bytes (keepalive noise / desynced stream) — raw chunks must not reset the idle clock
const garbage = Buffer.from('zz-no-frames-here'); // pure ASCII — never forms a 0x94 0xc3 frame header
for (let i = 0; i < 6; i++) {
await sleep(50); // ~300 ms wall time, well past the timeout
bridge.sockets[0].write(garbage);
}
svc.checkIdle();
await waitFor(() => bridge.sockets.length >= 2 && svc.status().connected);
} finally {
delete process.env.MESH_RECONNECT_MS;
delete process.env.MESH_IDLE_TIMEOUT_MS;
svc.onModuleDestroy();
bridge.close();
}
});
it('forces a reconnect when the undecoded frame buffer exceeds MESH_MAX_FRAME_BUFFER', async () => {
const bridge = await startBridge();
process.env.MESH_HOST = '127.0.0.1';
process.env.MESH_PORT = String(bridge.port);
process.env.MESH_RECONNECT_MS = '50'; // fast retry so the re-establishment is observable
process.env.MESH_MAX_FRAME_BUFFER = '100'; // tiny cap — Node delivers socket data in ~64 KB events, far below the 256_000 default
const svc = new MeshService();
try {
svc.onModuleInit();
await waitFor(
() => bridge.sockets.length === 1 && svc.status().connected,
);
// non-frame bytes (0x42 is not part of any frame header) — a desynced stream that never forms frames; the first ~64 KB data event alone exceeds the cap
bridge.sockets[0].write(Buffer.alloc(300_000, 0x42));
// overflow forced a reconnect — fresh session
await waitFor(() => bridge.sockets.length >= 2 && svc.status().connected);
// the new socket starts with a clean buffer and decodes valid frames normally (the myInfo frame is well under the cap)
const fr = create(P.Mesh.FromRadioSchema, {
payloadVariant: { case: 'myInfo', value: { myNodeNum: 777 } },
});
bridge.sockets[1].write(frame(toBinary(P.Mesh.FromRadioSchema, fr)));
await waitFor(() => svc.status().myNodeNum === 777);
} finally {
delete process.env.MESH_RECONNECT_MS;
delete process.env.MESH_MAX_FRAME_BUFFER;
svc.onModuleDestroy();
bridge.close();
}
});
it('inbound traffic keeps resetting the idle clock — no false trigger while packets arrive', async () => {
const bridge = await startBridge();
process.env.MESH_HOST = '127.0.0.1';
process.env.MESH_PORT = String(bridge.port);
process.env.MESH_IDLE_TIMEOUT_MS = '100';
const svc = new MeshService();
try {
svc.onModuleInit();
await waitFor(
() => bridge.sockets.length === 1 && svc.status().connected,
);
// steady drip of inbound frames for well over the timeout in wall time — each chunk resets lastPacketAt
for (let i = 0; i < 5; i++) {
await sleep(60);
bridge.sockets[0].write(nodeInfoFrame(42, kamilNodeInfo()));
}
svc.checkIdle(); // wall time is past the timeout, but traffic kept resetting it
await sleep(100); // enough for a destroy to surface as 'close' (connected would flip false)
expect(svc.status().connected).toBe(true);
expect(bridge.sockets.length).toBe(1);
} finally {
delete process.env.MESH_IDLE_TIMEOUT_MS;
svc.onModuleDestroy();
bridge.close();
}
});
it('does not trigger under the default timeout (MESH_IDLE_TIMEOUT_MS unset)', async () => {
const bridge = await startBridge();
process.env.MESH_HOST = '127.0.0.1';
process.env.MESH_PORT = String(bridge.port);
delete process.env.MESH_IDLE_TIMEOUT_MS; // default 60000 — a test can't wait that long, so silence must NOT fire
const svc = new MeshService();
try {
svc.onModuleInit();
await waitFor(
() => bridge.sockets.length === 1 && svc.status().connected,
);
await sleep(250); // silent, but far under the default budget
svc.checkIdle();
expect(svc.status().connected).toBe(true);
expect(bridge.sockets.length).toBe(1);
} finally {
delete process.env.MESH_IDLE_TIMEOUT_MS;
svc.onModuleDestroy();
bridge.close();
}
});
it('enables TCP keepalive on the bridge socket', async () => {
const setKeepAlive = vi.spyOn(net.Socket.prototype, 'setKeepAlive');
const bridge = await startBridge();
process.env.MESH_HOST = '127.0.0.1';
process.env.MESH_PORT = String(bridge.port);
const svc = new MeshService();
try {
svc.onModuleInit();
await waitFor(() => svc.status().connected);
expect(setKeepAlive).toHaveBeenCalledWith(true, 30_000);
} finally {
setKeepAlive.mockRestore();
svc.onModuleDestroy();
bridge.close();
}
});
});
describe('MeshService periodic autoreconnect', () => {
it('unconditionally reconnects when MESH_RECONNECT_INTERVAL_MS elapses', async () => {
const bridge = await startBridge();
process.env.MESH_HOST = '127.0.0.1';
process.env.MESH_PORT = String(bridge.port);
process.env.MESH_RECONNECT_MS = '50'; // fast retry so the re-establishment is observable
process.env.MESH_RECONNECT_INTERVAL_MS = '100'; // override — the default (300000) would never elapse in a test
const svc = new MeshService();
try {
svc.onModuleInit();
await waitFor(
() => bridge.sockets.length === 1 && svc.status().connected,
);
// interval fires → destroy → existing close/retry loop re-establishes with a fresh session
await waitFor(() => bridge.sockets.length >= 2 && svc.status().connected);
} finally {
delete process.env.MESH_RECONNECT_MS;
delete process.env.MESH_RECONNECT_INTERVAL_MS;
svc.onModuleDestroy();
bridge.close();
}
});
});
describe('MeshService send', () => { describe('MeshService send', () => {
it('sends on the test channel (index 3) by default, never primary', async () => { it('sends on the test channel (index 3) by default, never primary', async () => {
const bridge = await startBridge(); const bridge = await startBridge();
+68 -1
View File
@@ -81,6 +81,8 @@ export interface TelemetryRecord {
const HISTORY_LIMIT = 100; const HISTORY_LIMIT = 100;
/** Rows kept in SQLite across restarts; older rows are pruned on every poller tick. */ /** Rows kept in SQLite across restarts; older rows are pruned on every poller tick. */
const DB_RETENTION = 1000; const DB_RETENTION = 1000;
// how often the idle watchdog checks for a silent-but-connected bridge (ms)
const IDLE_CHECK_MS = 30_000;
// one row per MessageRecord, keyed by packet id (tx ids and injected ids are random uint32, rx uses the wire id) // one row per MessageRecord, keyed by packet id (tx ids and injected ids are random uint32, rx uses the wire id)
export const MESSAGES_DDL = `CREATE TABLE IF NOT EXISTS messages( export const MESSAGES_DDL = `CREATE TABLE IF NOT EXISTS messages(
@@ -208,6 +210,21 @@ export class MeshService implements OnModuleInit, OnModuleDestroy {
); );
// how often the poller re-upserts the history buffer into SQLite (ms) // how often the poller re-upserts the history buffer into SQLite (ms)
private readonly pollMs = parseInt(process.env.MESH_POLL_MS ?? '1000', 10); private readonly pollMs = parseInt(process.env.MESH_POLL_MS ?? '1000', 10);
// idle watchdog: force a reconnect when an ESTABLISHED connection stops decoding frames — the zombie-bridge case where outbound sends keep working but inbound frame flow is dead (raw non-frame bytes do NOT reset the clock). Read at construction like the other env config.
private readonly idleTimeoutMs = parseInt(
process.env.MESH_IDLE_TIMEOUT_MS ?? '60000',
10,
);
// lazy periodic autoreconnect: unconditionally destroy + re-establish every N ms so a fresh sync happens on schedule — the bridge/radio side tends to kill inbound RX within ~20 min of a fresh sync. Read at construction like the other env config.
private readonly reconnectIntervalMs = parseInt(
process.env.MESH_RECONNECT_INTERVAL_MS ?? '300000',
10,
);
// cap on undecoded frame buffer: a desynced stream that never forms frames would grow it without bound — force a reconnect instead. Env-overridable so tests can trigger it (Node caps socket 'data' events at ~64 KB).
private readonly maxFrameBuffer = parseInt(
process.env.MESH_MAX_FRAME_BUFFER ?? '256000',
10,
);
/** SQLite database backing the message history; null = persistence disabled (tests construct without one). */ /** SQLite database backing the message history; null = persistence disabled (tests construct without one). */
private db: DatabaseSync | null = null; private db: DatabaseSync | null = null;
@@ -255,6 +272,10 @@ export class MeshService implements OnModuleInit, OnModuleDestroy {
// telemetry history (newest kept, capped at HISTORY_LIMIT) // telemetry history (newest kept, capped at HISTORY_LIMIT)
private telemetryHistory: TelemetryRecord[] = []; private telemetryHistory: TelemetryRecord[] = [];
private reconnectTimer: NodeJS.Timeout | null = null; private reconnectTimer: NodeJS.Timeout | null = null;
/** epoch ms when the last valid frame was decoded from the bridge; reset on connect — idle watchdog baseline */
private lastPacketAt = Date.now();
private idleTimer: NodeJS.Timeout | null = null;
private periodicReconnectTimer: NodeJS.Timeout | null = null;
private shuttingDown = false; private shuttingDown = false;
onModuleInit(): void { onModuleInit(): void {
@@ -264,6 +285,15 @@ export class MeshService implements OnModuleInit, OnModuleDestroy {
// ponytail: full re-upsert of the ≤100-row buffer every tick — trivial load, zero bookkeeping // ponytail: full re-upsert of the ≤100-row buffer every tick — trivial load, zero bookkeeping
this.pollTimer = setInterval(() => this.flushHistory(), this.pollMs); this.pollTimer = setInterval(() => this.flushHistory(), this.pollMs);
} }
// idle watchdog: catches established-but-silent connections (zombie bridge) that the connect/retry loop never sees
this.idleTimer = setInterval(() => this.checkIdle(), IDLE_CHECK_MS);
// lazy periodic autoreconnect: unconditional fresh sync on a fixed schedule; when not connected, destroy is a no-op and the retry loop owns recovery
this.periodicReconnectTimer = setInterval(() => {
this.logger.log(
'periodic reconnect — MESH_RECONNECT_INTERVAL_MS elapsed',
);
this.socket?.destroy();
}, this.reconnectIntervalMs);
this.connect(); this.connect();
} }
@@ -271,6 +301,8 @@ export class MeshService implements OnModuleInit, OnModuleDestroy {
this.shuttingDown = true; this.shuttingDown = true;
if (this.reconnectTimer) clearTimeout(this.reconnectTimer); if (this.reconnectTimer) clearTimeout(this.reconnectTimer);
if (this.pollTimer) clearInterval(this.pollTimer); if (this.pollTimer) clearInterval(this.pollTimer);
if (this.idleTimer) clearInterval(this.idleTimer);
if (this.periodicReconnectTimer) clearInterval(this.periodicReconnectTimer);
this.socket?.destroy(); this.socket?.destroy();
this.db?.close(); this.db?.close();
} }
@@ -641,11 +673,17 @@ export class MeshService implements OnModuleInit, OnModuleDestroy {
private connect(): void { private connect(): void {
if (this.shuttingDown) return; if (this.shuttingDown) return;
const socket = net.createConnection({ host: this.host, port: this.port }); const socket = net.createConnection({ host: this.host, port: this.port });
// TCP-level keepalive: cheap insurance against a dead peer that the idle watchdog (application level) complements
socket.setKeepAlive(true, 30_000);
this.socket = socket; this.socket = socket;
socket.on('connect', () => { socket.on('connect', () => {
this.connected = true; this.connected = true;
// fresh connection — reset the idle clock so it can't self-trigger before any traffic arrives
this.lastPacketAt = Date.now();
// fresh sync on every (re)connect — drop stale channels/nodes/device info/telemetry from the old session; // fresh sync on every (re)connect — drop stale channels/nodes/device info/telemetry from the old session;
// message history is NOT reset: it's an append-only log backed by SQLite and survives reconnects // message history is NOT reset: it's an append-only log backed by SQLite and survives reconnects.
// The undecoded frame buffer goes too: leftover desync garbage must not re-trigger the overflow backstop on a healthy new stream.
this.buffer = new Uint8Array(0);
this.channelMap = new Map(); this.channelMap = new Map();
this.nodeMap = new Map(); this.nodeMap = new Map();
this.metadata = null; this.metadata = null;
@@ -666,9 +704,19 @@ export class MeshService implements OnModuleInit, OnModuleDestroy {
const merged = new Uint8Array(this.buffer.length + chunk.length); const merged = new Uint8Array(this.buffer.length + chunk.length);
merged.set(this.buffer, 0); merged.set(this.buffer, 0);
merged.set(chunk, this.buffer.length); merged.set(chunk, this.buffer.length);
// buffer-overflow backstop: a desynced stream that never forms frames would grow the buffer without bound — cap it and self-heal via reconnect
if (merged.length > this.maxFrameBuffer) {
this.logger.error(
'frame buffer overflow — stream desync, forcing reconnect',
);
socket.destroy();
return;
}
this.buffer = merged; this.buffer = merged;
const { packets, rest } = extractFrames(this.buffer); const { packets, rest } = extractFrames(this.buffer);
this.buffer = rest; this.buffer = rest;
// the watchdog tracks DECODED FRAMES, not raw bytes: non-frame noise (keepalives, desynced streams) must not keep a dead connection alive
if (packets.length) this.lastPacketAt = Date.now();
for (const p of packets) { for (const p of packets) {
decodeFromRadio(p) decodeFromRadio(p)
.then((fr) => this.handleFromRadio(fr)) .then((fr) => this.handleFromRadio(fr))
@@ -694,6 +742,25 @@ export class MeshService implements OnModuleInit, OnModuleDestroy {
this.reconnectTimer = setTimeout(() => this.connect(), this.reconnectMs); this.reconnectTimer = setTimeout(() => this.connect(), this.reconnectMs);
} }
/**
* Idle watchdog tick: force a reconnect when an ESTABLISHED connection has decoded no frames from
* the bridge for MESH_IDLE_TIMEOUT_MS — the zombie case where outbound sends keep working but inbound
* frame flow is dead (raw non-frame bytes arriving on the socket do not count). Destroying the socket
* routes through 'close' → onDown → scheduleReconnect, so the existing connect/retry loop re-establishes
* with a fresh sync; no new retry machinery here. Public so tests can drive a tick without waiting for
* the real interval (like flushHistory()).
*/
checkIdle(): void {
if (!this.connected) return; // disconnected state is owned by the connect/retry loop
const idle = Date.now() - this.lastPacketAt;
if (idle > this.idleTimeoutMs) {
this.logger.warn(
`no decoded bridge frames for ${idle} ms — forcing reconnect`,
);
this.socket?.destroy();
}
}
private async handleFromRadio( private async handleFromRadio(
fr: import('@meshtastic/protobufs').Mesh.FromRadio, fr: import('@meshtastic/protobufs').Mesh.FromRadio,
): Promise<void> { ): Promise<void> {
+28
View File
@@ -0,0 +1,28 @@
import { readdirSync, readFileSync } from 'node:fs';
import { join } from 'node:path';
import { fileURLToPath } from 'node:url';
import { describe, expect, it } from 'vitest';
const srcDir = fileURLToPath(new URL('.', import.meta.url));
function moduleFiles(dir: string): string[] {
return readdirSync(dir, { withFileTypes: true }).flatMap((entry) =>
entry.isDirectory()
? moduleFiles(join(dir, entry.name))
: entry.name.endsWith('.module.ts')
? [join(dir, entry.name)]
: [],
);
}
describe('ScheduleModule.forRoot single registration', () => {
// Regression guard (v0.21.7): each forRoot() call creates its own ScheduleExplorer that scans all providers, so a second call made every @Cron/@Interval job fire twice.
it('appears exactly once across all src/**/*.module.ts', () => {
const hits = moduleFiles(srcDir).flatMap((file) =>
(
readFileSync(file, 'utf8').match(/ScheduleModule\.forRoot\(/g) ?? []
).map(() => file.slice(srcDir.length)),
);
expect(hits).toHaveLength(1);
});
});