Merge pull request 'Add OpenAI-compatible LLM client service (v0.19.0)' (#40) from feat/llm-client into main

Add OpenAI-compatible LLM client service (v0.19.0) (#40)

AGENTS.md

@@ -28,3 +28,4 @@
 - MESHTASTIC.md — rules for Meshtastic communication (owner authority, language, response protocol); read before acting on mesh messages.
 - Hourly cron weather report job (`src/weather`): fetches Open-Meteo current conditions and broadcasts a one-line report to the mesh on the pinned send channel; env `WEATHER_CRON`/`WEATHER_LAT`/`WEATHER_LON`/`WEATHER_CITY`.
 - Channel-2 auto-responder (`src/commands`): event-driven — `CommandService.onRx()` is called by `MeshService` for every decoded rx text (right after the notify gate) and dispatches exact, case-sensitive `#command` matches on `COMMAND_CHANNEL` (env, default 2) through a single-registration-point registry (handlers registered in the constructor via `register(name, fn)`; handler gets `{ text, from?, id? }`, returns reply string or void = no reply). First command: `#ping` → replies `#pong!`. Replies go through `MeshService.respond(id, reply)`: auto-routed (rx → DM back to the original sender unless the original was a broadcast; tx → its destination), riding the original record's channel with `replyId = id` so the recipient app renders it as a threaded quote — `COMMAND_CHANNEL` is only the trigger/listen gate. Only when the trigger has no packet id does it fall back to legacy sendText routing on `COMMAND_CHANNEL`. A failing handler/respond only logs a warn. DI: provided by `CommandModule`'s factory (injects `MeshService`, attaches itself via `attachCommands()`) — no constructor-level cycle; `forwardRef` was tried and left one side null in NestJS 12's injector.
+- LLM client (`src/llm`): `LlmService` wraps the local OpenAI-compatible server (default `http://192.168.1.70:1111/v1`, no auth) with native fetch — plain feature module, no Mesh/Schedule dependency. Methods: `listModels()` → `data[].id`; `chat(messages, { maxTokens?, temperature? })` → `choices[0].message.content` (non-2xx throws with status + body snippet); `summarize(text)` = chat() with system prompt "Summarize the following text concisely." Env: `LLM_API_BASE`, `LLM_MODEL` (default `qwen3.8-27b`), `LLM_TIMEOUT_MS` (default 60000, via `AbortSignal.timeout`).

CHANGELOG.md

@@ -1,5 +1,12 @@
 # Changelog
 
+## v0.19.0 — 2026-09-25 18:03
+
+- New LLM client module (`src/llm`): `LlmService` speaks the OpenAI-compatible chat-completions API of the local server at `LLM_API_BASE` (default `http://192.168.1.70:1111/v1`, no auth) using native fetch only — no new dependencies. Plain feature module (`LlmModule`), wired into `app.module.ts`; no Mesh/Schedule dependency.
+- Methods: `listModels()` (GET `/models` → `data[].id` strings), `chat(messages, opts?)` (POST `/chat/completions` with `{ model, messages, ...opts }` → `choices[0].message.content`; non-2xx throws an Error carrying the HTTP status and a short body snippet; malformed responses without content throw too), and `summarize(text)` — thin wrapper over `chat()` with system prompt "Summarize the following text concisely." (the stated primary use).
+- Env: `LLM_API_BASE` (default `http://192.168.1.70:1111/v1`), `LLM_MODEL` (default `qwen3.8-27b`), `LLM_TIMEOUT_MS` (default `60000` — the local 27B model can be slow; applied via `AbortSignal.timeout`).
+- Tests: +4 specs — listModels parses ids, chat passes model/messages/opts through and returns content, chat throws on non-2xx with status in the message, summarize sends system prompt + user text.
+
 ## v0.18.1 — 2026-09-25 17:30
 
 - Command replies now ride `MeshService.respond(id, reply)` instead of direct `sendText` routing: when a handler returns a non-empty string and the trigger has a packet id, the reply is auto-routed (rx → DM back to the original sender unless the original was a broadcast; tx → its destination) on the **original message's channel** with `replyId = id`, so the recipient app renders it as a threaded quote of the triggering packet. `COMMAND_CHANNEL` is now only the trigger/listen gate — the reply channel comes from the record.

README.md

@@ -125,6 +125,9 @@ Config lives in `.env` (gitignored) and is passed via compose environment — cr
 - `MESH_DB_PATH` (default `./data/mesh-bot.db`) — SQLite file backing the message history (parent dir created if missing; in Docker, mounted at `/app/data` via a named volume so it survives container recreation)
 - `MESH_POLL_MS` (default `1000`) — how often the poller re-upserts the message history into SQLite and prunes rows beyond the newest 1000
 - `PORT` (default `3000`) — HTTP listen port
+- `LLM_API_BASE` (default `http://192.168.1.70:1111/v1`) — OpenAI-compatible LLM server base URL used by the in-process `LlmService` (`src/llm`; no auth)
+- `LLM_MODEL` (default `qwen3.8-27b`) — model id sent to `/chat/completions`
+- `LLM_TIMEOUT_MS` (default `60000`) — per-request timeout for LLM calls (the local 27B model can be slow)
 
 Note: the serial bridge accepts only ONE client at a time — if another app is
 connected to the radio, this bot will not get a connection.

package.json

@@ -1,6 +1,6 @@
 {
   "name": "mesh-bot",
-  "version": "0.18.1",
+  "version": "0.19.0",
   "description": "",
   "author": "",
   "private": true,

src/app.module.ts

@@ -1,10 +1,17 @@
 import { Module } from '@nestjs/common';
 import { BatteryAlertModule } from './battery/battery.module.js';
 import { CommandModule } from './commands/command.module.js';
+import { LlmModule } from './llm/llm.module.js';
 import { MeshModule } from './mesh/mesh.module.js';
 import { WeatherModule } from './weather/weather.module.js';
 
 @Module({
-  imports: [MeshModule, WeatherModule, BatteryAlertModule, CommandModule],
+  imports: [
+    MeshModule,
+    WeatherModule,
+    BatteryAlertModule,
+    CommandModule,
+    LlmModule,
+  ],
 })
 export class AppModule {}

src/llm/llm.module.ts

@@ -0,0 +1,8 @@
+import { Module } from '@nestjs/common';
+import { LlmService } from './llm.service.js';
+
+@Module({
+  providers: [LlmService],
+  exports: [LlmService],
+})
+export class LlmModule {}

src/llm/llm.service.spec.ts

@@ -0,0 +1,104 @@
+import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest';
+import type { ChatMessage } from './llm.service.js';
+import { LlmService } from './llm.service.js';
+
+const BASE = 'http://192.168.1.70:1111/v1';
+
+function json(body: unknown): Response {
+  return { ok: true, status: 200, json: async () => body } as Response;
+}
+
+/** Fresh service with a clean env (explicit base/model/timeout). */
+function makeService(): LlmService {
+  process.env.LLM_API_BASE = BASE;
+  process.env.LLM_MODEL = 'qwen3.8-27b';
+  process.env.LLM_TIMEOUT_MS = '60000';
+  return new LlmService();
+}
+
+describe('LlmService', () => {
+  let fetchMock: ReturnType<typeof vi.fn>;
+
+  beforeEach(() => {
+    fetchMock = vi.fn();
+    vi.stubGlobal('fetch', fetchMock);
+  });
+
+  afterEach(() => {
+    vi.unstubAllGlobals();
+    for (const k of ['LLM_API_BASE', 'LLM_MODEL', 'LLM_TIMEOUT_MS'])
+      delete process.env[k];
+  });
+
+  it('listModels returns the data[].id strings from GET /models', async () => {
+    const svc = makeService();
+    fetchMock.mockResolvedValueOnce(
+      json({
+        data: [
+          { id: 'qwen3.8-27b' },
+          { id: 'text-embedding-nomic-embed-text-v1.5' },
+        ],
+      }),
+    );
+
+    await expect(svc.listModels()).resolves.toEqual([
+      'qwen3.8-27b',
+      'text-embedding-nomic-embed-text-v1.5',
+    ]);
+    expect(fetchMock).toHaveBeenCalledTimes(1);
+    expect(fetchMock.mock.calls[0][0]).toBe(`${BASE}/models`);
+  });
+
+  it('chat posts model+messages and returns choices[0].message.content', async () => {
+    const svc = makeService();
+    fetchMock.mockResolvedValueOnce(
+      json({ choices: [{ message: { content: 'hello back' } }] }),
+    );
+
+    const messages: ChatMessage[] = [
+      { role: 'system', content: 'be brief' },
+      { role: 'user', content: 'hi' },
+    ];
+    await expect(
+      svc.chat(messages, { maxTokens: 50, temperature: 0.2 }),
+    ).resolves.toBe('hello back');
+
+    const [url, init] = fetchMock.mock.calls[0];
+    expect(url).toBe(`${BASE}/chat/completions`);
+    expect(init.method).toBe('POST');
+    expect(JSON.parse(init.body)).toEqual({
+      model: 'qwen3.8-27b',
+      messages,
+      maxTokens: 50,
+      temperature: 0.2,
+    });
+  });
+
+  it('chat throws on non-2xx with the HTTP status in the message', async () => {
+    const svc = makeService();
+    fetchMock.mockResolvedValueOnce({
+      ok: false,
+      status: 503,
+      text: async () => 'upstream down',
+    } as Response);
+
+    await expect(svc.chat([{ role: 'user', content: 'hi' }])).rejects.toThrow(
+      /HTTP 503.*upstream down/,
+    );
+  });
+
+  it('summarize calls chat with the system prompt plus the user text', async () => {
+    const svc = makeService();
+    fetchMock.mockResolvedValueOnce(
+      json({ choices: [{ message: { content: 'the summary' } }] }),
+    );
+
+    await expect(svc.summarize('a long text')).resolves.toBe('the summary');
+
+    const [, init] = fetchMock.mock.calls[0];
+    expect(JSON.parse(init.body).messages).toEqual([
+      { role: 'system', content: 'Summarize the following text concisely.' },
+      { role: 'user', content: 'a long text' },
+    ]);
+  });
+});

src/llm/llm.service.ts

@@ -0,0 +1,68 @@
+import { Injectable } from '@nestjs/common';
+
+/** A single chat message in OpenAI chat-completions shape. */
+export interface ChatMessage {
+  role: string;
+  content: string;
+}
+
+@Injectable()
+export class LlmService {
+  // read at construction, like other services' env config
+  private readonly apiBase =
+    process.env.LLM_API_BASE ?? 'http://192.168.1.70:1111/v1';
+  private readonly model = process.env.LLM_MODEL ?? 'qwen3.8-27b';
+  // local 27B model can be slow — generous default
+  private readonly timeoutMs = parseInt(
+    process.env.LLM_TIMEOUT_MS ?? '60000',
+    10,
+  );
+
+  /** GET {base}/models → the `data[].id` strings. */
+  async listModels(): Promise<string[]> {
+    const res = await fetch(`${this.apiBase}/models`, {
+      signal: AbortSignal.timeout(this.timeoutMs),
+    });
+    if (!res.ok) {
+      throw new Error(
+        `listModels failed: HTTP ${res.status} ${(await res.text()).slice(0, 200)}`,
+      );
+    }
+    const body = (await res.json()) as { data?: { id: string }[] };
+    return (body.data ?? []).map((m) => m.id);
+  }
+
+  /** POST {base}/chat/completions → `choices[0].message.content`. */
+  async chat(
+    messages: ChatMessage[],
+    opts?: { maxTokens?: number; temperature?: number },
+  ): Promise<string> {
+    const res = await fetch(`${this.apiBase}/chat/completions`, {
+      method: 'POST',
+      headers: { 'content-type': 'application/json' },
+      body: JSON.stringify({ model: this.model, messages, ...opts }),
+      signal: AbortSignal.timeout(this.timeoutMs),
+    });
+    if (!res.ok) {
+      throw new Error(
+        `chat failed: HTTP ${res.status} ${(await res.text()).slice(0, 200)}`,
+      );
+    }
+    const body = (await res.json()) as {
+      choices?: { message?: { content?: string } }[];
+    };
+    const content = body.choices?.[0]?.message?.content;
+    if (typeof content !== 'string') {
+      throw new Error('chat failed: no choices[0].message.content in response');
+    }
+    return content;
+  }
+
+  /** Thin wrapper over chat(): the stated primary use of this service. */
+  async summarize(text: string): Promise<string> {
+    return this.chat([
+      { role: 'system', content: 'Summarize the following text concisely.' },
+      { role: 'user', content: text },
+    ]);
+  }
+}