OpenCode/History
2
0
Fork 0
Code Issues 1 Pull Requests 1 Actions 1 Activity

Reduce turn latency: cap recent events (10→5) + concise-reasoning constraint (v0.9.0) #23

Merged
kamilkrzywda merged 1 commits from feat/turn-latency-hardening into master 2026-10-11 23:03:53 +02:00
Conversation 0 Commits 1 Files Changed 4
+46 -3
kamilkrzywda commented 2026-10-11 23:01:44 +02:00
Owner
Copy Link
Copy Source

Fixes #22

What

Reduced LLM turn latency with two prompt-level-only changes in server/engine.ts:

  • buildUserPrompt: cap recentEvents from the last 10 to the last 5 events (smaller input, less to reason over).
  • buildSystemPrompt: added a HARD CONSTRAINT nudging the reasoning model to keep its internal analysis brief before emitting the JSON (shortens the chain-of-thought — the 170–223s turn driver).

Reasoning stays enabled (reasoning_effort: low); only its length is limited. The deterministic stub, the TurnResult contract, max_tokens, and reasoning_effort are all unchanged.

Tests

  • Added 2 unit tests in server/engine.test.ts: (1) buildUserPrompt emits exactly the last 5 of N events, in order; (2) buildSystemPrompt includes the concise-reasoning constraint. 47 tests pass.
  • npm run typecheck, npm run lint, npm run build all green. e2e runs in CI.

Version

v0.8.0 → v0.9.0 (see CHANGELOG).

Fixes #22 ## What Reduced LLM turn latency with two **prompt-level-only** changes in `server/engine.ts`: - **`buildUserPrompt`**: cap `recentEvents` from the last 10 to the last 5 events (smaller input, less to reason over). - **`buildSystemPrompt`**: added a HARD CONSTRAINT nudging the reasoning model to keep its internal analysis brief before emitting the JSON (shortens the chain-of-thought — the 170–223s turn driver). Reasoning stays **enabled** (`reasoning_effort: low`); only its length is limited. The deterministic stub, the `TurnResult` contract, `max_tokens`, and `reasoning_effort` are all unchanged. ## Tests - Added 2 unit tests in `server/engine.test.ts`: (1) `buildUserPrompt` emits exactly the last 5 of N events, in order; (2) `buildSystemPrompt` includes the concise-reasoning constraint. **47 tests pass.** - `npm run typecheck`, `npm run lint`, `npm run build` all green. e2e runs in CI. ## Version v0.8.0 → v0.9.0 (see CHANGELOG).
kamilkrzywda added 1 commit 2026-10-11 23:01:44 +02:00
Reduce turn latency: cap recent events (10→5) + concise-reasoning constraint (v0.9.0)
CI / build + unit + e2e (pull_request) Successful in 2m9s
Details
e38cc5933f 
Fixes #22
kamilkrzywda scheduled this pull request to auto merge when all checks succeed 2026-10-11 23:02:33 +02:00
kamilkrzywda merged commit e70f65f112 into master 2026-10-11 23:03:53 +02:00
kamilkrzywda deleted branch feat/turn-latency-hardening 2026-10-11 23:03:53 +02:00
Sign in to join this conversation.
Reviewers
No Reviewers
Labels
No items
No labels
Milestone
No items
No Milestone
Assignees
Clear assignees
kamilkrzywda (Kamil Krzywda) smysakowski
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: OpenCode/History#23