Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
306756c63b | ||
|
|
af1c77fd54 | ||
|
|
73ce49c741 | ||
|
|
cdd52a3ddb | ||
|
|
172e7bed9c | ||
|
|
e2d712a26c |
@@ -12,7 +12,7 @@
|
||||
## Verification
|
||||
|
||||
1. **Local checks** — `npm run build` (nest build) and `npm test` (vitest; no e2e suite yet). Both must pass before a PR is opened.
|
||||
2. **Docker** — `docker compose build`, then after `docker compose up -d`: `curl -s -H "Host: bot.dev.kkhost.pl" http://localhost/api/status` returns JSON (`{ connected, myNodeNum, host, port }`). `"connected":false` is normal while the bridge/device is down — the service retries every 5 s.
|
||||
2. **Docker** — `docker compose build`, then after `docker compose up -d`: `curl -s -H "Host: bot.dev.kkhost.pl" http://localhost/api/status` returns JSON (`{ connected, myNodeNum, host, port }`). `"connected":false` is normal while the bridge/device is down — the service retries every 5 s. An idle watchdog also force-reconnects an ESTABLISHED-but-silent connection (zombie bridge: outbound works, inbound dead) when no packet arrives from the bridge for `MESH_IDLE_TIMEOUT_MS` ms (default 600000 = 10 min; checked every 30 s), and the socket has TCP keepalive enabled (30 s).
|
||||
|
||||
## API
|
||||
|
||||
@@ -27,5 +27,5 @@
|
||||
|
||||
- MESHTASTIC.md — rules for Meshtastic communication (owner authority, language, response protocol); read before acting on mesh messages.
|
||||
- Hourly cron weather report job (`src/weather`): fetches Open-Meteo current conditions and broadcasts a one-line report to the mesh on the pinned send channel; env `WEATHER_CRON`/`WEATHER_LAT`/`WEATHER_LON`/`WEATHER_CITY`.
|
||||
- Channel-2 auto-responder (`src/commands`): event-driven — `CommandService.onRx()` is called by `MeshService` for every decoded rx text (right after the notify gate) and dispatches exact, case-sensitive `#command` matches on `COMMAND_CHANNEL` (env, default 2) through a single-registration-point registry (handlers registered in the constructor via `register(name, fn)`; handler gets `{ text, from?, id? }`, returns reply string or void = no reply). First command: `#ping` → replies `#pong!`. Replies go through `MeshService.respond(id, reply)`: auto-routed (rx → DM back to the original sender unless the original was a broadcast; tx → its destination), riding the original record's channel with `replyId = id` so the recipient app renders it as a threaded quote — `COMMAND_CHANNEL` is only the trigger/listen gate. Only when the trigger has no packet id does it fall back to legacy sendText routing on `COMMAND_CHANNEL`. A failing handler/respond only logs a warn. DI: provided by `CommandModule`'s factory (injects `MeshService`, attaches itself via `attachCommands()`) — no constructor-level cycle; `forwardRef` was tried and left one side null in NestJS 12's injector. Prefix dispatch (`registerPrefix(name, fn)`) lets one handler own a whole family: the `#rss` subcommands live in `src/rss` (`RssService.handle`, registered as prefix `'rss'` from the CommandModule factory — exact matches still win over prefixes). Subcommands: `#rss help`; `#rss add <url> <name>` (validates URL, fetches with 15 s timeout, requires ≥1 titled item, rejects duplicate names case-insensitively; LLM generates 2-4 lowercase tags from up to 8 latest titles — on LLM failure the source is added with empty tags and the reply says so); `#rss remove <name>`; `#rss tags` (all distinct tags); `#rss sources`; `#rss get <name|#tag>` (LLM headline summary, maxTokens 200; `#tag` collects items across ALL sources carrying that tag — one dead feed does not sink the summary). Storage: `rss_sources(name TEXT PRIMARY KEY, url, tags, created_at)` table in the EXISTING MESH_DB_PATH file via a second node:sqlite connection (names matched case-insensitively); feed parsing via `fast-xml-parser` (RSS 2.0 + Atom; malformed/no-items → "not a valid feed"); LLM via injected `LlmService.chat()` with explicit maxTokens (50 tags / 200 summary) to keep mesh replies short.
|
||||
- Channel-2 auto-responder (`src/commands`): event-driven — `CommandService.onRx()` is called by `MeshService` for every decoded rx text (right after the notify gate) and dispatches exact, case-sensitive `#command` matches on `COMMAND_CHANNEL` (env, default 2) through a single-registration-point registry (handlers registered in the constructor via `register(name, fn)`; handler gets `{ text, from?, id? }`, returns a reply string — or an array of lines, sent as one message per line through the same routing — or void = no reply). First command: `#ping` → replies `#pong!`. Replies go through `MeshService.respond(id, reply)`: auto-routed (rx → DM back to the original sender unless the original was a broadcast; tx → its destination), riding the original record's channel with `replyId = id` so the recipient app renders it as a threaded quote — `COMMAND_CHANNEL` is only the trigger/listen gate. Only when the trigger has no packet id does it fall back to legacy sendText routing on `COMMAND_CHANNEL`. A failing handler/respond only logs a warn. DI: provided by `CommandModule`'s factory (injects `MeshService`, attaches itself via `attachCommands()`) — no constructor-level cycle; `forwardRef` was tried and left one side null in NestJS 12's injector. Prefix dispatch (`registerPrefix(name, fn)`) lets one handler own a whole family: the `#rss` subcommands live in `src/rss` (`RssService.handle`, registered as prefix `'rss'` from the CommandModule factory — exact matches still win over prefixes). Subcommands: `#rss help`; `#rss add <url> <name>` (validates URL, fetches with 15 s timeout, requires ≥1 titled item, rejects duplicate names case-insensitively; LLM generates 2-4 lowercase tags from up to 8 latest titles — on LLM failure the source is added with empty tags and the reply says so); `#rss remove <name>`; `#rss tags` (all distinct tags); `#rss sources`; `#rss get <name|#tag>` (LLM headline summary as 3-5 SEPARATE mesh messages: prompt asks for exactly 3-5 lines, one item per line, `<short header>: <one or two short sentences>` with ≤20-char header, no intro/outro/numbering; reply parsed into items — drop empty/separator-less lines, cap at 5, fewer than 3 → send what we got (no retry), zero → no message; each item sent via its own respond() threaded to the trigger packet id, per-line backstop `fitOneLine()` hard-cuts >185-char lines at a word boundary near 180 so every item fits one packet; maxTokens 2500 is headroom because the LLM server's reasoning tokens count against it — see rss.service.ts comment; `#tag` collects items across ALL sources carrying that tag — one dead feed does not sink the summary). Storage: `rss_sources(name TEXT PRIMARY KEY, url, tags, created_at)` table in the EXISTING MESH_DB_PATH file via a second node:sqlite connection (names matched case-insensitively); feed parsing via `fast-xml-parser` (RSS 2.0 + Atom; malformed/no-items → "not a valid feed"); LLM via injected `LlmService.chat()` with explicit maxTokens (50 tags / 2500 summary headroom).
|
||||
- LLM client (`src/llm`): `LlmService` wraps the local OpenAI-compatible server (default `http://192.168.1.70:1111/v1`, no auth) with native fetch — plain feature module, no Mesh/Schedule dependency. Methods: `listModels()` → `data[].id`; `chat(messages, { maxTokens?, temperature? })` → `choices[0].message.content` (non-2xx throws with status + body snippet); `summarize(text)` = chat() with system prompt "Summarize the following text concisely." Env: `LLM_API_BASE`, `LLM_MODEL` (default `qwen3.8-27b`), `LLM_TIMEOUT_MS` (default 60000, via `AbortSignal.timeout`).
|
||||
|
||||
@@ -1,5 +1,28 @@
|
||||
# Changelog
|
||||
|
||||
## v0.21.1 — 2026-09-26 06:47
|
||||
|
||||
- Bridge zombie-connection fix (production incident: outbound sends kept working ~10 h while inbound packets stopped entirely and `/api/status` stayed `connected:true`). New **idle watchdog**: the service tracks `lastPacketAt`, updated on EVERY chunk received from the bridge in the lowest-level socket `'data'` path (any frame kind, decodable or not) and reset when a connection successfully starts. A 30 s interval checks: if connected AND no inbound traffic for `MESH_IDLE_TIMEOUT_MS` (new env, default **600000** = 10 min), it logs WARN ("no bridge traffic for X ms — forcing reconnect") and destroys the socket — the EXISTING connect/retry loop (`MESH_RECONNECT_MS`) then re-establishes with a fresh sync. No changes to connect-error retry/backoff behavior; this only adds detection for established-but-silent connections.
|
||||
- TCP keepalive on the bridge socket: `socket.setKeepAlive(true, 30_000)` — cheap insurance against dead peers at the TCP level, complementing the application-level watchdog.
|
||||
- Tests: 165 → 169 — silence past an overridden timeout forces disconnect + reconnect (fake bridge), inbound traffic keeps resetting the idle clock so no false trigger, default timeout does not fire in a test window, and `setKeepAlive(true, 30_000)` is asserted on the socket.
|
||||
|
||||
## v0.21.0 — 2026-09-25 19:59
|
||||
|
||||
- `#rss get <name|#tag>` now replies with **3–5 SEPARATE mesh messages** instead of one small summary (owner request: the single-message summary was too small). Each item is its own packet, so no fragmentation — and each rides the trigger's channel threaded as a quote of the command packet.
|
||||
- New summarizer prompt asks for exactly 3-5 lines, one item per line, format `<short header>: <one or two short sentences>` with a ≤20-char header (e.g. `Culture: Museum opens new wing`), no intro/outro/numbering. maxTokens stays 2500 (reasoning-token headroom — see v0.20.1).
|
||||
- The LLM reply is parsed into items: split on newlines, trim, drop empty lines and lines without a `:` header separator, cap at 5. Fewer than 3 valid lines → send what we got (no failure, no retry — the model is non-deterministic); zero valid lines → no message sent.
|
||||
- Per-item backstop replaces the old whole-reply cut: any single line >185 chars is hard-cut at a word boundary near 180 (`fitOneLine`), so every item fits one packet even if the model ignores the prompt. `fitOnePacket()` (whole-reply ≤190) is gone — it no longer applies to joined text.
|
||||
- CommandService now sends array replies line by line: each element goes through the same routing as a single reply (`respond(id, line)` threaded to the trigger packet id; legacy sendText fallback per line when there is no packet id). A failing send stops the remaining lines (warn log), like any failed command.
|
||||
- LLM failure path unchanged: single `summary failed` reply via respond(). `#rss get #tag` gets the same multi-message treatment automatically (shared summary path — covered by tests). All other subcommands unchanged.
|
||||
- Tests: 158 → 165 — prompt shape + maxTokens asserted on the new call, 5-item cap, invalid-line filtering, fewer-than-3 fallback, per-line over-length cut through `handle()`, direct `parseSummaryItems`/`fitOneLine` specs, and CommandService array-reply routing (threaded respond ×N to one packet id; legacy sendText fallback).
|
||||
|
||||
## v0.20.1 — 2026-09-25 19:20
|
||||
|
||||
- `#rss get` summaries now fit ONE mesh packet (≤190 chars). Mesh fragments longer replies into fire-and-forget packets that recipients frequently never receive, and the old prompt ("a few short one-line highlights") produced ~326-334 char replies.
|
||||
- The summary prompt now asks for "at most 3 short bullet lines, each under 45 characters … bullets only - no intro, no other text" (verified against the live LLM: exactly 3 compliant bullets, ~110 chars). A `fitOnePacket()` backstop additionally cuts any over-length reply at complete-bullet boundaries so delivery is one packet even if the model ignores the prompt.
|
||||
- The summary `chat()` call now uses an explicit `maxTokens = 2500` (was 200). Empirically verified on this server: qwen3.8-27b emits `reasoning_content` and those tokens COUNT against max_tokens (cap 150 → completion_tokens 150, of which 105 were reasoning, finish_reason 'length'; a cap of 500 was fully consumed by reasoning with zero visible content). Thinking length for this task is unbounded and uncontrollable server-side (`/no_think`, `chat_template_kwargs.enable_thinking` and `thinking_budget` are all ignored) — observed 32..1776 tokens across runs. 2500 covers every observed run plus ~80 content tokens, and keeps the worst case (~42 s at ~60 tok/s) under the 60 s LLM timeout. The old cap of 200 guaranteed empty replies whenever reasoning exceeded it.
|
||||
- Tests: 154 → 158 — prompt shape + maxTokens asserted on the summary call, over-length reply cut through `handle()`, and direct `fitOnePacket` specs (pass-through at/under 190, bullet-boundary cut, hard-cut fallback for non-bulleted paragraphs).
|
||||
|
||||
## v0.20.0 — 2026-09-25 18:40
|
||||
|
||||
- New `#rss` command family on the command channel (ch2): `#rss help`, `#rss add <url> <name>` (validates URL + fetches with a 15 s timeout, requires ≥1 item with a non-empty title, rejects duplicate names case-insensitively; tags are generated by the LLM from up to 8 latest titles — on LLM failure the source is still added with empty tags and the reply says so), `#rss remove <name>`, `#rss tags` (all distinct tags), `#rss sources`, `#rss get <name>` and `#rss get #<tag>` (LLM headline summary, maxTokens 200; tag form collects items across ALL sources carrying the tag — one dead feed does not sink the summary). Bare `#rss` / unknown subcommand → help. All replies ride the existing respond() path like `#ping`.
|
||||
|
||||
+1
-1
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "mesh-bot",
|
||||
"version": "0.20.0",
|
||||
"version": "0.21.1",
|
||||
"description": "",
|
||||
"author": "",
|
||||
"private": true,
|
||||
|
||||
@@ -114,6 +114,30 @@ describe('CommandService.onRx', () => {
|
||||
expect(mesh.respond).toHaveBeenCalledWith(5, 'up from 99');
|
||||
});
|
||||
|
||||
it('sends each line of an array reply as its own threaded respond() to the same packet id', async () => {
|
||||
const [svc, mesh] = makeService();
|
||||
svc.register('multi', () => ['one: first item', 'two: second item']);
|
||||
await svc.onRx(2, '#multi', 42, 9);
|
||||
expect(mesh.respond).toHaveBeenCalledTimes(2);
|
||||
expect(mesh.respond.mock.calls[0]).toEqual([9, 'one: first item']);
|
||||
expect(mesh.respond.mock.calls[1]).toEqual([9, 'two: second item']);
|
||||
expect(mesh.sendText).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('sends each line of an array reply via the legacy sendText fallback when there is no packet id', async () => {
|
||||
const [svc, mesh] = makeService();
|
||||
svc.register('multi', () => ['one: first item', 'two: second item']);
|
||||
await svc.onRx(2, '#multi', 42); // no id → legacy path on COMMAND_CHANNEL, DM back to the sender
|
||||
expect(mesh.sendText).toHaveBeenCalledTimes(2);
|
||||
expect(mesh.sendText.mock.calls[0]).toEqual([
|
||||
42,
|
||||
'one: first item',
|
||||
undefined,
|
||||
2,
|
||||
]);
|
||||
expect(mesh.respond).not.toHaveBeenCalled();
|
||||
});
|
||||
|
||||
it('swallows a failing handler (a bad command must not crash the app)', async () => {
|
||||
const [svc, mesh] = makeService();
|
||||
svc.register('boom', () => {
|
||||
|
||||
@@ -14,10 +14,10 @@ export interface CommandContext {
|
||||
id?: number;
|
||||
}
|
||||
|
||||
/** A command handler: returns the reply text to send back, or void/undefined for no reply. */
|
||||
/** A command handler: returns the reply text(s) to send back — an array sends one message per line — or void/undefined for no reply. */
|
||||
export type CommandHandler = (
|
||||
ctx: CommandContext,
|
||||
) => string | undefined | Promise<string | undefined>;
|
||||
) => string | string[] | undefined | Promise<string | string[] | undefined>;
|
||||
|
||||
@Injectable()
|
||||
export class CommandService {
|
||||
@@ -83,18 +83,21 @@ export class CommandService {
|
||||
try {
|
||||
const reply = await handler({ text: trimmed, from, id });
|
||||
if (reply === undefined || reply === null) return; // void = no reply
|
||||
// an array reply sends one message per line, all threaded to the same trigger packet id
|
||||
for (const line of Array.isArray(reply) ? reply : [reply]) {
|
||||
if (id !== undefined && this.mesh) {
|
||||
// respond() auto-routes the reply: rides the original record's channel with replyId = id (threaded quote)
|
||||
await this.mesh.respond(id, reply);
|
||||
// respond() auto-routes each line: rides the original record's channel with replyId = id (threaded quote)
|
||||
await this.mesh.respond(id, line);
|
||||
} else {
|
||||
// defensive fallback when the trigger has no packet id: legacy routing on COMMAND_CHANNEL
|
||||
await this.mesh?.sendText(
|
||||
from ?? BROADCAST_ADDR,
|
||||
reply,
|
||||
line,
|
||||
undefined,
|
||||
this.commandChannel,
|
||||
);
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
this.logger.warn(
|
||||
`[commands] '${trimmed}' failed: ${(err as Error).message}`,
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { create, fromBinary, toBinary } from '@bufbuild/protobuf';
|
||||
import * as P from '@meshtastic/protobufs';
|
||||
import { BadRequestException, NotFoundException } from '@nestjs/common';
|
||||
import * as net from 'node:net';
|
||||
import { DatabaseSync } from 'node:sqlite';
|
||||
import { beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import { BROADCAST_ADDR, extractFrames, frame } from './codec.js';
|
||||
@@ -249,6 +250,107 @@ describe('MeshService connect', () => {
|
||||
});
|
||||
});
|
||||
|
||||
const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms));
|
||||
|
||||
describe('MeshService idle watchdog', () => {
|
||||
it('forces a reconnect when an established connection goes silent past MESH_IDLE_TIMEOUT_MS', async () => {
|
||||
const bridge = await startBridge();
|
||||
process.env.MESH_HOST = '127.0.0.1';
|
||||
process.env.MESH_PORT = String(bridge.port);
|
||||
process.env.MESH_RECONNECT_MS = '50'; // fast retry so the re-establishment is observable
|
||||
process.env.MESH_IDLE_TIMEOUT_MS = '100'; // override — the default (600000) would never fire in a test
|
||||
const svc = new MeshService();
|
||||
try {
|
||||
svc.onModuleInit();
|
||||
await waitFor(
|
||||
() => bridge.sockets.length === 1 && svc.status().connected,
|
||||
);
|
||||
|
||||
// zombie case: connected but no inbound traffic at all — wait past the timeout, then drive a watchdog tick
|
||||
await sleep(250);
|
||||
svc.checkIdle();
|
||||
|
||||
// socket torn down → existing connect/retry loop re-establishes with a fresh session
|
||||
await waitFor(() => bridge.sockets.length >= 2 && svc.status().connected);
|
||||
} finally {
|
||||
delete process.env.MESH_RECONNECT_MS;
|
||||
delete process.env.MESH_IDLE_TIMEOUT_MS;
|
||||
svc.onModuleDestroy();
|
||||
bridge.close();
|
||||
}
|
||||
});
|
||||
|
||||
it('inbound traffic keeps resetting the idle clock — no false trigger while packets arrive', async () => {
|
||||
const bridge = await startBridge();
|
||||
process.env.MESH_HOST = '127.0.0.1';
|
||||
process.env.MESH_PORT = String(bridge.port);
|
||||
process.env.MESH_IDLE_TIMEOUT_MS = '100';
|
||||
const svc = new MeshService();
|
||||
try {
|
||||
svc.onModuleInit();
|
||||
await waitFor(
|
||||
() => bridge.sockets.length === 1 && svc.status().connected,
|
||||
);
|
||||
|
||||
// steady drip of inbound frames for well over the timeout in wall time — each chunk resets lastPacketAt
|
||||
for (let i = 0; i < 5; i++) {
|
||||
await sleep(60);
|
||||
bridge.sockets[0].write(nodeInfoFrame(42, kamilNodeInfo()));
|
||||
}
|
||||
|
||||
svc.checkIdle(); // wall time is past the timeout, but traffic kept resetting it
|
||||
await sleep(100); // enough for a destroy to surface as 'close' (connected would flip false)
|
||||
expect(svc.status().connected).toBe(true);
|
||||
expect(bridge.sockets.length).toBe(1);
|
||||
} finally {
|
||||
delete process.env.MESH_IDLE_TIMEOUT_MS;
|
||||
svc.onModuleDestroy();
|
||||
bridge.close();
|
||||
}
|
||||
});
|
||||
|
||||
it('does not trigger under the default timeout (MESH_IDLE_TIMEOUT_MS unset)', async () => {
|
||||
const bridge = await startBridge();
|
||||
process.env.MESH_HOST = '127.0.0.1';
|
||||
process.env.MESH_PORT = String(bridge.port);
|
||||
delete process.env.MESH_IDLE_TIMEOUT_MS; // default 600000 — a test can't wait that long, so silence must NOT fire
|
||||
const svc = new MeshService();
|
||||
try {
|
||||
svc.onModuleInit();
|
||||
await waitFor(
|
||||
() => bridge.sockets.length === 1 && svc.status().connected,
|
||||
);
|
||||
|
||||
await sleep(250); // silent, but far under the default budget
|
||||
svc.checkIdle();
|
||||
|
||||
expect(svc.status().connected).toBe(true);
|
||||
expect(bridge.sockets.length).toBe(1);
|
||||
} finally {
|
||||
delete process.env.MESH_IDLE_TIMEOUT_MS;
|
||||
svc.onModuleDestroy();
|
||||
bridge.close();
|
||||
}
|
||||
});
|
||||
|
||||
it('enables TCP keepalive on the bridge socket', async () => {
|
||||
const setKeepAlive = vi.spyOn(net.Socket.prototype, 'setKeepAlive');
|
||||
const bridge = await startBridge();
|
||||
process.env.MESH_HOST = '127.0.0.1';
|
||||
process.env.MESH_PORT = String(bridge.port);
|
||||
const svc = new MeshService();
|
||||
try {
|
||||
svc.onModuleInit();
|
||||
await waitFor(() => svc.status().connected);
|
||||
expect(setKeepAlive).toHaveBeenCalledWith(true, 30_000);
|
||||
} finally {
|
||||
setKeepAlive.mockRestore();
|
||||
svc.onModuleDestroy();
|
||||
bridge.close();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('MeshService send', () => {
|
||||
it('sends on the test channel (index 3) by default, never primary', async () => {
|
||||
const bridge = await startBridge();
|
||||
|
||||
@@ -81,6 +81,8 @@ export interface TelemetryRecord {
|
||||
const HISTORY_LIMIT = 100;
|
||||
/** Rows kept in SQLite across restarts; older rows are pruned on every poller tick. */
|
||||
const DB_RETENTION = 1000;
|
||||
// how often the idle watchdog checks for a silent-but-connected bridge (ms)
|
||||
const IDLE_CHECK_MS = 30_000;
|
||||
|
||||
// one row per MessageRecord, keyed by packet id (tx ids and injected ids are random uint32, rx uses the wire id)
|
||||
export const MESSAGES_DDL = `CREATE TABLE IF NOT EXISTS messages(
|
||||
@@ -208,6 +210,11 @@ export class MeshService implements OnModuleInit, OnModuleDestroy {
|
||||
);
|
||||
// how often the poller re-upserts the history buffer into SQLite (ms)
|
||||
private readonly pollMs = parseInt(process.env.MESH_POLL_MS ?? '1000', 10);
|
||||
// idle watchdog: force a reconnect when an ESTABLISHED connection goes silent — the zombie-bridge case where outbound sends keep working but inbound packets stop entirely. Read at construction like the other env config.
|
||||
private readonly idleTimeoutMs = parseInt(
|
||||
process.env.MESH_IDLE_TIMEOUT_MS ?? '600000',
|
||||
10,
|
||||
);
|
||||
|
||||
/** SQLite database backing the message history; null = persistence disabled (tests construct without one). */
|
||||
private db: DatabaseSync | null = null;
|
||||
@@ -255,6 +262,9 @@ export class MeshService implements OnModuleInit, OnModuleDestroy {
|
||||
// telemetry history (newest kept, capped at HISTORY_LIMIT)
|
||||
private telemetryHistory: TelemetryRecord[] = [];
|
||||
private reconnectTimer: NodeJS.Timeout | null = null;
|
||||
/** epoch ms of the last inbound chunk from the bridge (any frame kind); reset on connect — idle watchdog baseline */
|
||||
private lastPacketAt = Date.now();
|
||||
private idleTimer: NodeJS.Timeout | null = null;
|
||||
private shuttingDown = false;
|
||||
|
||||
onModuleInit(): void {
|
||||
@@ -264,6 +274,8 @@ export class MeshService implements OnModuleInit, OnModuleDestroy {
|
||||
// ponytail: full re-upsert of the ≤100-row buffer every tick — trivial load, zero bookkeeping
|
||||
this.pollTimer = setInterval(() => this.flushHistory(), this.pollMs);
|
||||
}
|
||||
// idle watchdog: catches established-but-silent connections (zombie bridge) that the connect/retry loop never sees
|
||||
this.idleTimer = setInterval(() => this.checkIdle(), IDLE_CHECK_MS);
|
||||
this.connect();
|
||||
}
|
||||
|
||||
@@ -271,6 +283,7 @@ export class MeshService implements OnModuleInit, OnModuleDestroy {
|
||||
this.shuttingDown = true;
|
||||
if (this.reconnectTimer) clearTimeout(this.reconnectTimer);
|
||||
if (this.pollTimer) clearInterval(this.pollTimer);
|
||||
if (this.idleTimer) clearInterval(this.idleTimer);
|
||||
this.socket?.destroy();
|
||||
this.db?.close();
|
||||
}
|
||||
@@ -641,9 +654,13 @@ export class MeshService implements OnModuleInit, OnModuleDestroy {
|
||||
private connect(): void {
|
||||
if (this.shuttingDown) return;
|
||||
const socket = net.createConnection({ host: this.host, port: this.port });
|
||||
// TCP-level keepalive: cheap insurance against a dead peer that the idle watchdog (application level) complements
|
||||
socket.setKeepAlive(true, 30_000);
|
||||
this.socket = socket;
|
||||
socket.on('connect', () => {
|
||||
this.connected = true;
|
||||
// fresh connection — reset the idle clock so it can't self-trigger before any traffic arrives
|
||||
this.lastPacketAt = Date.now();
|
||||
// fresh sync on every (re)connect — drop stale channels/nodes/device info/telemetry from the old session;
|
||||
// message history is NOT reset: it's an append-only log backed by SQLite and survives reconnects
|
||||
this.channelMap = new Map();
|
||||
@@ -663,6 +680,8 @@ export class MeshService implements OnModuleInit, OnModuleDestroy {
|
||||
this.logger.log(`requested initial sync (want_config_id=${nonce})`);
|
||||
});
|
||||
socket.on('data', (chunk) => {
|
||||
// lowest-level receive path: every inbound chunk counts as bridge traffic, decodable or not — this is what the idle watchdog watches
|
||||
this.lastPacketAt = Date.now();
|
||||
const merged = new Uint8Array(this.buffer.length + chunk.length);
|
||||
merged.set(this.buffer, 0);
|
||||
merged.set(chunk, this.buffer.length);
|
||||
@@ -694,6 +713,22 @@ export class MeshService implements OnModuleInit, OnModuleDestroy {
|
||||
this.reconnectTimer = setTimeout(() => this.connect(), this.reconnectMs);
|
||||
}
|
||||
|
||||
/**
|
||||
* Idle watchdog tick: force a reconnect when an ESTABLISHED connection has seen no inbound bridge
|
||||
* traffic for MESH_IDLE_TIMEOUT_MS — the zombie case where outbound sends keep working but inbound
|
||||
* packets stop. Destroying the socket routes through 'close' → onDown → scheduleReconnect, so the
|
||||
* existing connect/retry loop re-establishes with a fresh sync; no new retry machinery here.
|
||||
* Public so tests can drive a tick without waiting for the real interval (like flushHistory()).
|
||||
*/
|
||||
checkIdle(): void {
|
||||
if (!this.connected) return; // disconnected state is owned by the connect/retry loop
|
||||
const idle = Date.now() - this.lastPacketAt;
|
||||
if (idle > this.idleTimeoutMs) {
|
||||
this.logger.warn(`no bridge traffic for ${idle} ms — forcing reconnect`);
|
||||
this.socket?.destroy();
|
||||
}
|
||||
}
|
||||
|
||||
private async handleFromRadio(
|
||||
fr: import('@meshtastic/protobufs').Mesh.FromRadio,
|
||||
): Promise<void> {
|
||||
|
||||
+122
-11
@@ -5,7 +5,7 @@ import { DatabaseSync } from 'node:sqlite';
|
||||
import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
|
||||
import type { LlmService } from '../llm/llm.service.js';
|
||||
import { RssStore } from './rss.store.js';
|
||||
import { RssService } from './rss.service.js';
|
||||
import { fitOneLine, parseSummaryItems, RssService } from './rss.service.js';
|
||||
|
||||
const FEED_A = `<rss version="2.0"><channel>
|
||||
<item><title>Alpha headline</title></item>
|
||||
@@ -149,18 +149,79 @@ describe('RssService.handle', () => {
|
||||
expect(await svc.handle(ctx('#rss sources'))).toBe('Aye, Bee');
|
||||
});
|
||||
|
||||
it('get <name>: summarizes the latest titles via chat with a bounded maxTokens', async () => {
|
||||
it('get <name>: summarizes via chat with a bounded maxTokens and replies with headered lines', async () => {
|
||||
stored().add({ name: 'myfeed', url: 'https://a.example/rss', tags: [] });
|
||||
fetchMock.mockResolvedValueOnce(feedResponse(FEED_A));
|
||||
llm.chat.mockResolvedValueOnce('Alpha and beta happened.');
|
||||
llm.chat.mockResolvedValueOnce(
|
||||
'Chips: EU funds foundries.\nPark: Riverfront expansion approved.',
|
||||
);
|
||||
|
||||
const reply = await svc.handle(ctx('#rss get myfeed'));
|
||||
|
||||
expect(reply).toBe('Alpha and beta happened.');
|
||||
expect(reply).toEqual([
|
||||
'Chips: EU funds foundries.',
|
||||
'Park: Riverfront expansion approved.',
|
||||
]);
|
||||
const [messages, opts] = llm.chat.mock.calls[0];
|
||||
expect(opts).toEqual({ maxTokens: 200 });
|
||||
expect(messages[0].content).toContain('Alpha headline');
|
||||
expect(messages[0].content).toContain('Beta headline');
|
||||
// headroom for the server's reasoning tokens (they count against max_tokens) — see rss.service.ts
|
||||
expect(opts).toEqual({ maxTokens: 2500 });
|
||||
const prompt = messages[0].content;
|
||||
expect(prompt).toContain('exactly 3-5 lines');
|
||||
expect(prompt).toContain('<short header>: <one or two short sentences>');
|
||||
expect(prompt).toContain('at most 20 characters');
|
||||
expect(prompt).toContain('No intro, no outro, no numbering');
|
||||
expect(prompt).toContain('Alpha headline');
|
||||
expect(prompt).toContain('Beta headline');
|
||||
});
|
||||
|
||||
it('get <name>: caps items at 5', async () => {
|
||||
stored().add({ name: 'myfeed', url: 'https://a.example/rss', tags: [] });
|
||||
fetchMock.mockResolvedValueOnce(feedResponse(FEED_A));
|
||||
const seven = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
|
||||
.map((c) => `${c}: item ${c}`)
|
||||
.join('\n');
|
||||
llm.chat.mockResolvedValueOnce(seven);
|
||||
|
||||
const reply = await svc.handle(ctx('#rss get myfeed'));
|
||||
|
||||
expect(reply).toEqual([
|
||||
'a: item a',
|
||||
'b: item b',
|
||||
'c: item c',
|
||||
'd: item d',
|
||||
'e: item e',
|
||||
]);
|
||||
});
|
||||
|
||||
it('get <name>: drops empty and separator-less lines, sends what remains when fewer than 3 are valid', async () => {
|
||||
stored().add({ name: 'myfeed', url: 'https://a.example/rss', tags: [] });
|
||||
fetchMock.mockResolvedValueOnce(feedResponse(FEED_A));
|
||||
llm.chat.mockResolvedValueOnce(
|
||||
'Chips: EU funds foundries.\n\nJust a sentence without a header\n \nPark: expansion approved.',
|
||||
);
|
||||
|
||||
const reply = await svc.handle(ctx('#rss get myfeed'));
|
||||
|
||||
expect(reply).toEqual([
|
||||
'Chips: EU funds foundries.',
|
||||
'Park: expansion approved.',
|
||||
]);
|
||||
});
|
||||
|
||||
it('get <name>: hard-cuts an over-length line at a word boundary so each item fits one packet', async () => {
|
||||
stored().add({ name: 'myfeed', url: 'https://a.example/rss', tags: [] });
|
||||
fetchMock.mockResolvedValueOnce(feedResponse(FEED_A));
|
||||
const longLine = `Chips: ${'word '.repeat(40)}`; // 207 chars — over the per-line cap
|
||||
llm.chat.mockResolvedValueOnce(`${longLine}\nPark: short line.`);
|
||||
|
||||
const reply = await svc.handle(ctx('#rss get myfeed'));
|
||||
|
||||
expect(Array.isArray(reply)).toBe(true);
|
||||
const [cut, ok] = reply as string[];
|
||||
expect(cut.length).toBeLessThanOrEqual(180);
|
||||
expect(longLine.startsWith(cut)).toBe(true); // prefix — cut at a word boundary, not mid-word
|
||||
expect(cut.endsWith(' ')).toBe(false);
|
||||
expect(ok).toBe('Park: short line.'); // other lines untouched
|
||||
});
|
||||
|
||||
it('get #tag: collects items across all sources with the tag; one dead feed does not sink it', async () => {
|
||||
@@ -171,11 +232,11 @@ describe('RssService.handle', () => {
|
||||
.mockResolvedValueOnce(feedResponse(FEED_A)) // A answers (Alpha, Beta)
|
||||
.mockRejectedValueOnce(new Error('down')); // B is dead
|
||||
|
||||
llm.chat.mockResolvedValueOnce('Only alpha survived.');
|
||||
llm.chat.mockResolvedValueOnce('Chips: only alpha survived.');
|
||||
const reply = await svc.handle(ctx('#rss get #tech'));
|
||||
|
||||
// B's fetch rejected, yet the summary still went out from A's titles alone
|
||||
expect(reply).toBe('Only alpha survived.');
|
||||
expect(reply).toEqual(['Chips: only alpha survived.']);
|
||||
expect(llm.chat).toHaveBeenCalledTimes(1);
|
||||
const content = llm.chat.mock.calls[0][0][0].content;
|
||||
expect(content).toContain('Alpha headline');
|
||||
@@ -188,9 +249,15 @@ describe('RssService.handle', () => {
|
||||
.mockResolvedValueOnce(feedResponse(FEED_A)) // A → Alpha, Beta
|
||||
.mockResolvedValueOnce(feedResponse(FEED_B)); // B → Gamma
|
||||
|
||||
llm.chat.mockResolvedValueOnce('all three.');
|
||||
await svc.handle(ctx('#rss get #tech'));
|
||||
llm.chat.mockResolvedValueOnce(
|
||||
'Alpha: first headline.\nGamma: second headline.',
|
||||
);
|
||||
const reply = await svc.handle(ctx('#rss get #tech'));
|
||||
|
||||
expect(reply).toEqual([
|
||||
'Alpha: first headline.',
|
||||
'Gamma: second headline.',
|
||||
]);
|
||||
const content = llm.chat.mock.calls[0][0][0].content;
|
||||
expect(content).toContain('Alpha headline');
|
||||
expect(content).toContain('Gamma headline'); // proves B's feed was collected too
|
||||
@@ -204,3 +271,47 @@ describe('RssService.handle', () => {
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('parseSummaryItems (per-line items for multi-message summaries)', () => {
|
||||
it('keeps lines with a header separator, drops empty and separator-less ones', () => {
|
||||
expect(
|
||||
parseSummaryItems(
|
||||
'Chips: EU funds foundries.\n\nNo header here\n \nPark: expansion approved.',
|
||||
),
|
||||
).toEqual(['Chips: EU funds foundries.', 'Park: expansion approved.']);
|
||||
});
|
||||
|
||||
it('caps at 5 items', () => {
|
||||
const seven = ['a', 'b', 'c', 'd', 'e', 'f', 'g']
|
||||
.map((c) => `${c}: item`)
|
||||
.join('\n');
|
||||
expect(parseSummaryItems(seven)).toHaveLength(5);
|
||||
expect(parseSummaryItems(seven)[4]).toBe('e: item');
|
||||
});
|
||||
|
||||
it('returns [] when nothing is valid', () => {
|
||||
expect(parseSummaryItems('no headers at all\n')).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('fitOneLine (one mesh packet per line)', () => {
|
||||
it('passes through lines that already fit, including exactly 185 chars', () => {
|
||||
const short = 'Chips: EU funds foundries.';
|
||||
expect(fitOneLine(short)).toBe(short);
|
||||
const exact = 'x'.repeat(185);
|
||||
expect(fitOneLine(exact)).toBe(exact);
|
||||
});
|
||||
|
||||
it('cuts over-long lines at a word boundary near 180', () => {
|
||||
const long = `Chips: ${'word '.repeat(40)}`; // 207 chars
|
||||
const cut = fitOneLine(long);
|
||||
expect(cut.length).toBeLessThanOrEqual(180);
|
||||
expect(long.startsWith(cut)).toBe(true); // prefix — not mid-word
|
||||
expect(cut.endsWith(' ')).toBe(false);
|
||||
});
|
||||
|
||||
it('hard-cuts at 180 when there is no word boundary in reach', () => {
|
||||
const long = 'w'.repeat(250);
|
||||
expect(fitOneLine(long)).toBe('w'.repeat(180));
|
||||
});
|
||||
});
|
||||
|
||||
+41
-9
@@ -10,12 +10,24 @@ import { RssStore } from './rss.store.js';
|
||||
const FETCH_TIMEOUT_MS = 15000; // feed fetches — same budget as the opencode notifier's POSTs
|
||||
const MAX_TITLES = 8; // "up to 8 latest item titles" per the design
|
||||
const TAG_MAX_TOKENS = 50;
|
||||
const SUMMARY_MAX_TOKENS = 200;
|
||||
// The LLM server (qwen3.8-27b) emits reasoning_content and those tokens COUNT against max_tokens
|
||||
// (verified: cap 150 → completion_tokens 150 of which 105 were reasoning, finish_reason 'length').
|
||||
// Thinking length for this task is unbounded and uncontrollable server-side (/no_think,
|
||||
// chat_template_kwargs.enable_thinking and thinking_budget are all ignored) — observed 32..1776
|
||||
// tokens across runs. So the cap must be headroom for reasoning + ~80 content tokens:
|
||||
// 2500 covers every observed run (worst case ≈42 s at ~60 tok/s, under LLM_TIMEOUT_MS=60 s).
|
||||
const SUMMARY_MAX_TOKENS = 2500;
|
||||
// Mesh fragments replies longer than ~190 chars into fire-and-forget packets that recipients
|
||||
// often never get — each summary line must fit ONE packet on its own. The prompt asks for short
|
||||
// headered lines; this is the hard backstop if the model ignores it (cut at a word boundary).
|
||||
const MAX_LINE_CHARS = 185; // per-line one-packet threshold
|
||||
const LINE_CUT_AT = 180; // where over-long lines get cut, snapped back to a word boundary
|
||||
const MAX_ITEMS = 5; // cap on summary items sent as separate messages
|
||||
|
||||
/**
|
||||
* The `#rss` command family: stored RSS sources (SQLite, shared MESH_DB_PATH file) with
|
||||
* LLM-generated tags and on-demand headline summaries. All replies are short strings —
|
||||
* CommandService routes them through MeshService.respond() like any other command reply.
|
||||
* LLM-generated tags and on-demand headline summaries. Replies are short strings — or an array
|
||||
* of lines for multi-message summaries, which CommandService sends one respond() per line.
|
||||
*/
|
||||
@Injectable()
|
||||
export class RssService {
|
||||
@@ -29,8 +41,8 @@ export class RssService {
|
||||
this.store = new RssStore(db);
|
||||
}
|
||||
|
||||
/** Entry point for the `#rss` prefix — always replies (errors included). */
|
||||
async handle(ctx: CommandContext): Promise<string> {
|
||||
/** Entry point for the `#rss` prefix — always replies (errors included); `get` may reply with several lines. */
|
||||
async handle(ctx: CommandContext): Promise<string | string[]> {
|
||||
const words = ctx.text.trim().split(/\s+/);
|
||||
switch (words[1]) {
|
||||
case undefined: // bare `#rss`
|
||||
@@ -131,8 +143,8 @@ export class RssService {
|
||||
return names.length ? names.join(', ') : 'no sources yet';
|
||||
}
|
||||
|
||||
/** `#rss get <name>` or `#rss get #<tag>`. */
|
||||
private async get(target: string): Promise<string> {
|
||||
/** `#rss get <name>` or `#rss get #<tag>` — replies with 3-5 separate one-packet lines. */
|
||||
private async get(target: string): Promise<string | string[]> {
|
||||
if (!target) return 'usage: #rss get <name|#tag>';
|
||||
|
||||
let urls: string[];
|
||||
@@ -163,15 +175,18 @@ export class RssService {
|
||||
if (!titles.length) return 'no items found';
|
||||
|
||||
try {
|
||||
return await this.llm.chat(
|
||||
const reply = await this.llm.chat(
|
||||
[
|
||||
{
|
||||
role: 'user',
|
||||
content: `Summarize these RSS headlines into a few short one-line highlights. Be concise:\n${titles.join('\n')}`,
|
||||
content: `Summarize these RSS headlines into exactly 3-5 lines, one item per line, each in the format "<short header>: <one or two short sentences>" where the header is at most 20 characters (e.g. "Culture: Museum opens new wing"). No intro, no outro, no numbering - items only:\n${titles.join('\n')}`,
|
||||
},
|
||||
],
|
||||
{ maxTokens: SUMMARY_MAX_TOKENS },
|
||||
);
|
||||
// each item goes out as its own mesh message — CommandService sends the array line by line;
|
||||
// fewer than 3 valid lines is fine (the model is non-deterministic), we send what we got
|
||||
return parseSummaryItems(reply).map(fitOneLine);
|
||||
} catch (err) {
|
||||
this.logger.warn(`[rss] summary failed: ${(err as Error).message}`);
|
||||
return 'summary failed';
|
||||
@@ -190,6 +205,23 @@ export class RssService {
|
||||
}
|
||||
}
|
||||
|
||||
/** Parse an LLM summary into per-line items: drop empty lines and lines without a `:` header separator, cap at MAX_ITEMS. */
|
||||
export function parseSummaryItems(reply: string): string[] {
|
||||
return reply
|
||||
.split('\n')
|
||||
.map((l) => l.trim())
|
||||
.filter((l) => l && l.includes(':'))
|
||||
.slice(0, MAX_ITEMS);
|
||||
}
|
||||
|
||||
/** Hard-cut a single summary line at a word boundary near LINE_CUT_AT when it exceeds MAX_LINE_CHARS. */
|
||||
export function fitOneLine(line: string): string {
|
||||
if (line.length <= MAX_LINE_CHARS) return line;
|
||||
const slice = line.slice(0, LINE_CUT_AT);
|
||||
const at = slice.lastIndexOf(' ');
|
||||
return (at > 0 ? slice.slice(0, at) : slice).trimEnd();
|
||||
}
|
||||
|
||||
/** Normalize LLM tag output: lowercase, no #, whitespace → '-', deduped. */
|
||||
export function sanitizeTags(raw: string): string[] {
|
||||
const seen = new Set<string>();
|
||||
|
||||
Reference in New Issue
Block a user