Rewire MeshService off the raw TCP socket + codec onto the Phase-1
MeshMonitorClient contract (HttpMeshMonitorClient in production,
FakeMeshMonitorClient in tests): status/messages/nodes/channels/telemetry
are polled (15s/1s/30s/5min/60s) instead of streamed, outbound broadcasts
and DMs go through client.send(), respond/inject/notifier/command/battery/
weather behavior and all /api/* + MCP endpoint shapes are preserved.
The fake TCP bridge is deleted and every consumer spec (mesh service, MCP
controller, command auto-responder, battery alerts) now drives the real
service through the widened fake client. Docker compose drops
MESH_HOST/MESH_PORT and adds the MESHMONITOR_* env (token from .env,
self-signed TLS via NODE_TLS_REJECT_UNAUTHORIZED=0).
Live smoke against https://meshmonitor.kkhost.pl: getStatus ->
{connected:true, myNodeNum:3994646728, longName:'SP8MK Bot',
nodeResponsive:true}; getNodes -> 104 nodes.
Build green, suite 17 files / 245 tests (baseline 234).
Phase 1 of the MeshMonitor HTTP-gateway migration: a stateless REST client
that a later phase wires into MeshService to replace the raw
Meshtastic protobuf-over-TCP bridge.
- mesh-monitor-client.ts: MeshMonitorClient interface, HttpMeshMonitorClient
(native fetch, Bearer auth, injectable base/source/poll params), mappers for
the gateway's actual JSON shapes (source-scoped paths, flat node position,
second-based node lastHeard vs ms message/telemetry timestamps, long-format
telemetry, packet id = 3rd segment of the id string), MeshMonitorError,
nodeNumToId/nodeIdToNum, shared assertSendTarget XOR, and
createMeshMonitorClientFromEnv() (null when MESHMONITOR_API_TOKEN unset).
- fake-mesh-monitor-client.ts: in-memory double mirroring the client contract
(set*/pushInbound config, sends[] recorder, outbound mirrored into
getMessages, newest-first, limit/channel/toNodeId/fromNodeId filters).
- meshmonitor.module.ts: thin Nest provider exposing the env-built client
(or null) under MESH_MONITOR_CLIENT for Phase 2's MeshModule.
- specs: HttpMeshMonitorClient against an in-process node:http gateway
(raw->typed mapping per endpoint, Authorization: Bearer header, send body
shape with channel/toNodeId XOR + replyId passthrough, 401/413/429/503
-> MeshMonitorError); FakeMeshMonitorClient behavior.
No MeshService changes, no version bump, no changelog, no PR.
A 0% batteryLevel is an invalid/no-data value (no battery wired or
misconfigured node), not a genuinely critical battery, so it must not
produce a "⚠️ battery low" alert. Add a non-positive guard to
shouldAlert after the undefined check: 0 (and any non-positive) now
skips, while 1-20 still alert and >20 still doesn't. Threshold/cooldown/
batching unchanged.
Tests: 3 new specs in the check-threshold block (0% excluded from the
batch, lone 0% sends nothing, 1% still alerts). 180 -> 183.
CodeNomad's per-workspace instance proxy was upgraded and now only
serves opencode v2 /api/* routes; the v1 /session/... routes 403, which
broke all notifications.
- Discovery: GET {api}{proxyPath}/api/session (v2 {data,cursor} envelope)
instead of GET {base}/session; sessions filtered by location.directory
and the max time.updated entry is picked.
- Send: POST {api}{proxyPath}/api/session/{sid}/prompt with a {text}
body instead of POST {base}/session/{id}/message with {parts:[...]}.
- v2 returns 2xx immediately, so the 15 s read-timeout-as-delivered
branch is kept only as a harmless safety net.
- Updated the spec to the v2 routes/body/envelope (incl. a
directory-filter assertion) and kept AGENTS.md in sync.
Owner request: 'now add #help with short list of all command'. Registered in
the CommandService constructor next to #ping; returns a SINGLE multi-line
string (one packet, ~203 chars) — not an array, which would drip-feed one
spaced packet per line through the v0.21.5 MESH_MULTI_SEND_DELAY_MS loop.
Also hardened the v0.21.5 multi-send-spacing spec: its wall-clock assertion
('first send < 8 ms after t0') flaked under full-suite load — replaced with
fake timers, which assert deterministically that line 1 is out before any time
elapses and lines 2-3 land exactly one delay apart.
Production incident: '#rss get lublin' returned an array of lines and
CommandService sent each one back-to-back within the same second — on the real
mesh 3 of 4 items were dropped in RF (the owner saw only the last one).
CommandService.onRx now waits MESH_MULTI_SEND_DELAY_MS (new env, default 5000 =
5 s, read at construction like the other timing knobs) between consecutive
lines of an array reply — no delay before the first line. The delay sits in the
shared per-line loop, so both the respond() path and the legacy sendText
fallback are covered by one await.
Production incident: '#rss get lublin' at 9:41 AM failed with 'The operation
was aborted due to timeout' — the LlmService call hit its 60 s cap while the
local OpenAI-compatible server (qwen3.8-27b, uncontrollable reasoning tokens)
was slow/busy. The same endpoint answers a trivial prompt in ~1 s when idle,
so this is latency variance on the server side, not a dead endpoint.
Bump the LLM_TIMEOUT_MS fallback from 60000 to 180000 (3 min headroom for
slow reasoning periods); still env-overridable, only the default changed.
The v0.21.2 frame-based watchdog works in production (it fired today: 'no
decoded bridge frames for 628474 ms — forcing reconnect', and the reconnect
delivered queued packets), but the bridge/radio side at 192.168.1.4 keeps
killing inbound RX within ~20 min of a fresh sync, repeatedly. Two tuning
changes requested by the owner:
- MESH_IDLE_TIMEOUT_MS default lowered from 600000 (10 min) to 60000 (1 min).
- New lazy periodic autoreconnect: MESH_RECONNECT_INTERVAL_MS (new env,
default 300000 = 5 min, read at construction like the others). An interval
started in onModuleInit unconditionally logs and destroys the socket so the
existing close -> scheduleReconnect loop re-establishes with a fresh sync on
schedule; cleared in onModuleDestroy. When not connected at tick time,
destroy is a no-op and the retry loop owns recovery — no special-casing.
Tests: 171 -> 172 — periodic reconnect observed end-to-end with an overridden
short interval (fake bridge disconnect + re-establish); existing watchdog tests
pass unchanged, comments updated to the new 60 s default.
Second production zombie slipped past v0.21.1's watchdog: text+telemetry RX
dead for >1 h while TX worked and the watchdog never fired. Root cause proven
at runtime via /proc/net/snmp — InSegs delta = 2 segments per 30 s (OutSegs 0):
the bridge peer keeps dribbling small NON-frame bytes (~every 15 s, keepalive
noise or a desynced stream) and v0.21.1 updated lastPacketAt on EVERY socket
chunk, so 'no traffic for MESH_IDLE_TIMEOUT_MS' was never true even though zero
valid Meshtastic frames decoded.
- The 'data' handler no longer updates lastPacketAt per chunk; it is updated
only when extractFrames yields >=1 valid packet (the reset in the 'connect'
handler is unchanged), so a stream that decodes nothing trips the watchdog
after MESH_IDLE_TIMEOUT_MS even while raw bytes keep arriving.
- Buffer-overflow backstop: if the undecoded frame buffer exceeds
MESH_MAX_FRAME_BUFFER (new env, default 256000) — e.g. one data event of a
desynced stream that never forms frames — log ERROR and destroy the socket;
the existing close -> scheduleReconnect loop self-heals it. The undecoded
buffer is also reset on every (re)connect so leftover garbage from a dead
session can't re-trigger the backstop or corrupt the first frames of a
healthy new stream.
Tests: 169 -> 171 — raw non-frame bytes do NOT keep the watchdog alive
(garbage-only stream forces disconnect + reconnect, today's incident shape),
and an undecoded chunk over MESH_MAX_FRAME_BUFFER triggers destroy + a fresh
session that decodes normally (cap env-overridden to 100 since Node delivers
socket data in ~64 KB events, far below the 256_000 default).
Production incident: the bridge TCP connection went zombie — outbound sends
kept working for ~10 h while inbound packets stopped entirely, and
/api/status stayed connected:true because nothing detects inbound silence on
an established connection.
- Track lastPacketAt: updated on every chunk received from the bridge in the
lowest-level socket 'data' path (any frame kind, decodable or not) and reset
when a connection successfully starts, so a fresh connection can't
self-trigger.
- Idle watchdog: a 30 s interval checks connected AND now - lastPacketAt >
MESH_IDLE_TIMEOUT_MS (new env, default 600000 = 10 min) — logs WARN with the
idle duration and destroys the socket so the EXISTING connect/retry loop
re-establishes it with a fresh sync. No changes to connect-error retry or
backoff; this only adds detection for established-but-silent connections.
- TCP keepalive: socket.setKeepAlive(true, 30_000) on the bridge socket —
cheap insurance against dead peers at the TCP level.
Tests (fake-bridge based): silence past an overridden timeout forces a
disconnect + reconnect; inbound traffic keeps resetting the idle clock so no
false trigger; the default timeout does not fire in a test window; and
setKeepAlive(true, 30_000) is asserted on the socket. checkIdle() is public so
tests drive a tick without waiting for the real interval (like flushHistory()).
The single-message summary was too small (owner request). The summarizer
prompt now asks for exactly 3-5 lines, one item per line, in the format
'<short header>: <one or two short sentences>' with a <=20-char header,
no intro/outro/numbering. maxTokens stays 2500 (reasoning-token headroom).
The reply is parsed into items: drop empty lines and lines without a ':'
header separator, cap at 5; fewer than 3 valid lines sends what we got
(no failure/retry), zero valid lines sends nothing. Each item goes out as
its own mesh message via respond(id, line) - all threaded to the trigger
packet id on its channel. Per-item backstop fitOneLine() hard-cuts any
line >185 chars at a word boundary near 180 so every item fits one packet;
the old whole-reply fitOnePacket() cut is gone.
CommandService sends array replies line by line through the same routing
as single replies (respond per line, or legacy sendText fallback per line
when there is no packet id). LLM failure path unchanged: single
'summary failed' reply.
Replies >~190 chars fragment into fire-and-forget packets recipients
often never receive; old prompt produced ~326-334 char replies.
Prompt now asks for at most 3 short bullet lines, each under 45 chars,
bullets only (verified live: exactly 3 compliant bullets, ~110 chars).
fitOnePacket() backstop cuts any over-length reply at complete-bullet
boundaries so delivery is one packet even if the model ignores the prompt.
maxTokens 200 -> 2500 on the summary chat(): empirically verified that
this server's reasoning tokens count against max_tokens (cap 150 ->
completion 150 of which 105 were reasoning; cap 500 fully consumed by
reasoning, zero visible content) and thinking is uncontrollable
server-side (/no_think, chat_template_kwargs.enable_thinking,
thinking_budget all ignored), observed 32..1776 tokens for this task.
2500 covers every observed run + ~80 content tokens, worst case ~42 s
under the 60 s LLM timeout.
#rss subcommands on the command channel: help / add <url> <name> /
remove <name> / tags / sources / get <name|#tag>. Sources are stored in
a new rss_sources table in the existing MESH_DB_PATH file (second
node:sqlite connection, case-insensitive names); feeds parsed with
fast-xml-parser (RSS 2.0 + Atom); AI tags and headline summaries via
injected LlmService.chat() with bounded maxTokens (50/200).
CommandService gains prefix registration so one handler owns the whole
#rss family while #ping stays an exact match; wired in CommandModule's
factory — still no constructor-level cycle, no forwardRef. One dead feed
does not sink a tag summary.
LlmService wraps the local OpenAI-compatible server (default
http://192.168.1.70:1111/v1, no auth) with native fetch only — no new
dependencies. listModels() -> data[].id; chat(messages, opts?) ->
choices[0].message.content (non-2xx throws with status + body snippet);
summarize(text) = chat() with a concise-summary system prompt.
Env: LLM_API_BASE / LLM_MODEL (qwen3.8-27b) / LLM_TIMEOUT_MS (60000,
AbortSignal.timeout). Plain feature module wired into app.module.ts;
docs synced (AGENTS.md, README env list).
Event-driven CommandService dispatches exact case-sensitive #command
matches on COMMAND_CHANNEL (default 2) through a single-registration-point
registry; first command #ping -> #pong!. Replies ride COMMAND_CHANNEL:
directed DM back to the sender when present, else broadcast. Failing
handler/send only logs a warn.
DI: CommandModule's factory injects MeshService and attaches itself via
MeshService.attachCommands() — a constructor-level cycle leaves one side
null in NestJS 12's injector (verified), so no forwardRef is used.
Tests 106 -> 121; docs synced (AGENTS.md, MESHTASTIC.md, README).
kamilkrzywda