Merge pull request 'v0.59.0 — SSE frames drop terminal package history (#133)' (#134) from feat/sse-payload into master

CHANGELOG.md

@@ -1,5 +1,9 @@
 # Changelog
 
+## v0.59.0 — 2026-09-23 05:48
+- **SSE frames no longer carry package history (ISS-133).** Measured on the live instance first: each `/api/events` frame was compact JSON at ~597 KB, pushed once per real second regardless of time scale (the background tick is a fixed 1 s interval; ×1000 just advances the clock further per frame) — and 99.8 % of it was `packages`: 1,521 rows because terminal packages are never deleted from the DB (expired 564, delivered 528, ignored 386, abandoned 31), while only 12 were `open`. No component ever reads a package with a terminal status — the market list renders `open`, and on-board cargo is `loaded`/`expired` (`LeftPane`, `cargoWeightFor`) — so the history was pure re-sent weight. New `sseFrame(snapshot)` in `lib/game/state.ts` projects a snapshot down to the client-rendered statuses (`SSE_PACKAGE_STATUSES = open|loaded|expired`); both SSE paths use it — the broadcast in `lib/sim/engine.ts` and the initial frame in `app/api/events/route.ts`, so a connect never costs one full-history frame. Live payload drops ~597 KB → ~10 KB per frame (≈ 60×), bounded forever by fleet/market size instead of growing with play time. `/api/state` and action responses keep the FULL package set unchanged, so every e2e oracle that asserts `delivered`/`abandoned` transitions via state lookups still works; the polling fallback stays a strict superset of what SSE pushes. Deliberately NOT done: hash-skip broadcasts (the snapshot's `clockMs` advances on every tick even while idle — `advanceMs = min(elapsed, 2000 ms) × scale ≥ 1 s` — so no two frames are ever equal), throttling (already exactly 1 Hz), and a diff/patch engine.
+- **Tests.** Unit +3 in new `lib/game/state.test.ts`: the projection keeps exactly open/loaded/expired (order preserved, terminal rows dropped), passes account/simState/planes through untouched without mutating its input, and pins the status set itself. E2E: `e2e/events.spec.ts` now records every package status seen in any frame plus the largest frame — asserts no terminal status ever appears and frames stay under 200 KB (a full-history regression at ~597 KB fails loudly).
+
 ## v0.58.0 — 2026-09-22 21:27
 - **Plane markers no longer teleport across the antimeridian (ISS-124).** The follow-up to ISS-120's known limitation: a plane mid-crossing still used the snapshot's wrapped lat/lon (`positionAt` wraps every longitude into [−180, 180]), so its marker jumped +179.x → −179.x between polls — a visible 360° teleport on the map while its route line stayed continuous. `planesGeoJson` in `lib/mapData.ts` now takes the airport index (`airportsByIcao`, same parameter shape as `routesGeoJson`) and re-expresses each FLYING plane's longitude against its departure airport — the same anchor `greatCircleArc` keeps for its first sample — via a new module-private `unwrapLon(lon, prev)` (the ±360 shift until |Δlon| ≤ 180 that `unwrapLons`'s loop body already used; `unwrapLons` now calls it). The unwrap applies only when BOTH leg endpoints exist in the index, mirroring `routesGeoJson`'s guard exactly: a marker is unwrapped if and only if its line is drawn. Grounded planes, repairing planes and flying planes with a missing endpoint keep their raw wrapped position (unchanged behavior); non-crossing legs are byte-identical (no |Δlon| > 180 → the shift never fires). `components/Map.tsx` passes `airportsIndexRef.current ?? new Map()` at both call sites (`applyDynamic` and the source bootstrap in `ensureLayers`) — planes draw immediately at their wrapped position until the airport index loads, then unwrap.
 - **Tests.** Unit +6 in `lib/mapData.test.ts` (new "antimeridian marker unwrap" block on the real NZAA/NFTF pair): eastward crossing unwraps a mid-leg −179.5 to +180.5 against the origin's +174.8, westward (reversed pair) unwraps below −180, and — the key invariant — every sample of `greatCircleArc(NZAA→NFTF)` fed back as a WRAPPED snapshot position comes out exactly on that arc's own sample, so marker and line agree frame to frame; grounded planes keep their raw coordinate, a flying plane with a missing endpoint stays wrapped (no line drawn either), and non-crossing legs are untouched. E2E +1 in `e2e/map-interactions.spec.ts`: the starter is relocated to Auckland via a fewest-legs chain over `/api/airports` (each hop ≤ 3400 km — within the ATR72's guaranteed range even in worst-case headwind; the tank is topped for free with the `set-fuel` hook before every dispatch so auto-refuel costs nothing), then a hook-created NZAA → NFTF package is loaded and flown at scale 1000 while the marker's longitude is sampled from the planes source every ~500 ms (real time) until the server reports landing — no two consecutive samples may differ by ≥ 180° (the teleport), some sample must sit on the east world copy (lon > 180, proving the crossing was actually traversed unwrapped rather than skipped), and the sampled span must exceed 4°. The plane chains back home afterwards so the other specs start from the canonical state.

app/api/events/route.ts

@@ -1,9 +1,11 @@
-import { getStateSnapshot } from '@/lib/game/state';
+import { getStateSnapshot, sseFrame } from '@/lib/game/state';
 import { subscribe } from '@/lib/sse';
 
-// ISS-129 (v0.57.0): Server-Sent Events feed of full state snapshots — the current snapshot on
-// connect, then one after every sim tick (broadcast from lib/sim/engine.ts). One-way push; player
-// actions stay REST POSTs. Heartbeats keep idle proxies from dropping the connection.
+// ISS-129 (v0.57.0): Server-Sent Events feed of state snapshots — the current snapshot on connect,
+// then one after every sim tick (broadcast from lib/sim/engine.ts). One-way push; player actions
+// stay REST POSTs. Heartbeats keep idle proxies from dropping the connection. ISS-133 (v0.59.0):
+// frames carry the sseFrame projection (terminal package history dropped) — same shape as the
+// pushed frames, so a connect never costs one full-history frame.
 
 const encoder = new TextEncoder();
 const HEARTBEAT_MS = 15_000;
@@ -39,7 +41,7 @@ export async function GET() {
               // same as above
             }
           }, HEARTBEAT_MS);
-          controller.enqueue(encoder.encode(`data: ${JSON.stringify(snapshot)}\n\n`));
+          controller.enqueue(encoder.encode(`data: ${JSON.stringify(sseFrame(snapshot))}\n\n`));
         })
         .catch(() => {
           cleanup();

e2e/events.spec.ts

@@ -8,8 +8,13 @@ const baseUrl = process.env.E2E_BASE_URL ?? 'https://flight.dev.kkhost.pl';
 
 interface StateBody {
   simState: { clockMs: number };
+  packages?: Array<{ status: string }>;
 }
 
+// ISS-133 (v0.59.0): frames carry only the package statuses the client renders — terminal history
+// (delivered/abandoned/ignored) is unbounded in the DB and must never leak into a pushed frame.
+const ALLOWED_PACKAGE_STATUSES = new Set(['open', 'loaded', 'expired']);
+
 test('GET /api/events streams state snapshots over SSE', async () => {
   const res = await fetch(`${baseUrl}/api/events`, { headers: { Accept: 'text/event-stream' } });
   expect(res.status).toBe(200);
@@ -20,6 +25,8 @@ test('GET /api/events streams state snapshots over SSE', async () => {
   const decoder = new TextDecoder();
   let buffer = '';
   const clocks: number[] = []; // simState.clockMs of each parsed snapshot event
+  const statusesSeen = new Set<string>(); // package statuses observed in any frame (ISS-133)
+  let maxFrameBytes = 0;
 
   try {
     // Collect two snapshot events: the initial one, then at least one pushed by a background tick.
@@ -46,6 +53,8 @@ test('GET /api/events streams state snapshots over SSE', async () => {
           if (!line.startsWith('data: ')) continue; // heartbeats are comment lines (" : ping")
           const snap = JSON.parse(line.slice('data: '.length)) as StateBody;
           clocks.push(snap.simState.clockMs);
+          for (const p of snap.packages ?? []) statusesSeen.add(p.status);
+          maxFrameBytes = Math.max(maxFrameBytes, line.length);
         }
       }
     }
@@ -56,4 +65,11 @@ test('GET /api/events streams state snapshots over SSE', async () => {
   // The initial snapshot arrives on connect; the second comes from a background tick within ~1 s.
   expect(clocks.length).toBeGreaterThanOrEqual(2);
   expect(clocks[1]).toBeGreaterThan(clocks[0]); // clock advanced between pushes — real updates, not replays
+
+  // ISS-133: no terminal package history in any frame (the DB keeps it forever; frames must not),
+  // and a bounded frame size even on an instance with long-running history (~597 KB pre-v0.59.0).
+  for (const status of statusesSeen) {
+    expect(ALLOWED_PACKAGE_STATUSES.has(status)).toBe(true);
+  }
+  expect(maxFrameBytes).toBeLessThan(200_000);
 });

lib/game/state.test.ts

@@ -0,0 +1,61 @@
+import { describe, expect, it } from 'vitest';
+import { SSE_PACKAGE_STATUSES, sseFrame, type PackageState, type StateSnapshot } from './state';
+
+// ISS-133 (v0.59.0) — the SSE projection keeps only the package statuses any component renders.
+
+const pkg = (status: PackageState['status'], id: string): PackageState => ({
+  id,
+  originIcao: 'LEMD',
+  originName: 'Madrid–Barajas',
+  destIcao: 'EHAM',
+  destName: 'Amsterdam Schiphol',
+  reward: 1000,
+  cargoType: 'mail',
+  cargoCount: 3,
+  weightKg: 900,
+  special: null,
+  distanceKm: 2000,
+  deadlineSimTime: 1_000_000,
+  status,
+  planeId: null,
+  createdSimTime: 0,
+});
+
+const snapshot = (packages: PackageState[]): StateSnapshot => ({
+  account: { cash: 20_000, reputation: 1 },
+  simState: { clockMs: 42 },
+  catalog: [],
+  fuelPricePerL: 0.8,
+  sellFraction: 0.5,
+  planes: [],
+  packages,
+});
+
+describe('sseFrame', () => {
+  it('keeps exactly the open/loaded/expired packages and drops terminal history', () => {
+    const all = [
+      pkg('open', 'p-open'),
+      pkg('delivered', 'p-delivered'),
+      pkg('loaded', 'p-loaded'),
+      pkg('abandoned', 'p-abandoned'),
+      pkg('expired', 'p-expired'),
+      pkg('ignored', 'p-ignored'),
+    ];
+    const frame = sseFrame(snapshot(all));
+    expect(frame.packages.map((p) => p.id)).toEqual(['p-open', 'p-loaded', 'p-expired']);
+  });
+
+  it('passes the rest of the snapshot through untouched and does not mutate its input', () => {
+    const all = [pkg('open', 'a'), pkg('delivered', 'b')];
+    const snap = snapshot(all);
+    const frame = sseFrame(snap);
+    expect(frame.account).toEqual({ cash: 20_000, reputation: 1 });
+    expect(frame.simState).toEqual({ clockMs: 42 });
+    expect(frame.planes).toEqual([]);
+    expect(all.map((p) => p.status)).toContain('delivered'); // input intact
+  });
+
+  it('SSE_PACKAGE_STATUSES matches what the client renders (open market, on-board cargo)', () => {
+    expect([...SSE_PACKAGE_STATUSES].sort()).toEqual(['expired', 'loaded', 'open']);
+  });
+});

lib/game/state.ts

@@ -85,6 +85,16 @@ export interface StateSnapshot {
   sellFraction: number;
 }
 
+// ISS-133 (v0.59.0): the SSE projection of a snapshot — only the package statuses any component
+// renders: open (market list), loaded + expired (on-board cargo, which keeps its payload slot).
+// Terminal history (delivered/abandoned/ignored) is unbounded in the DB and never read by the
+// client, so it stays out of every pushed frame; /api/state and action responses keep the full set.
+export const SSE_PACKAGE_STATUSES = ['open', 'loaded', 'expired'] as const;
+
+export function sseFrame(snapshot: StateSnapshot): StateSnapshot {
+  return { ...snapshot, packages: snapshot.packages.filter((p) => (SSE_PACKAGE_STATUSES as readonly string[]).includes(p.status)) };
+}
+
 export async function getStateSnapshot(): Promise<StateSnapshot> {
   const { db } = await getGameDb();
   // NOTE: use plain select(), not the relational db.query.* API — in drizzle-orm

lib/sim/engine.ts

@@ -7,7 +7,7 @@ import { haversineKm, initialBearingDeg } from '@/lib/geo';
 import { weatherFor } from '@/lib/weather';
 import { FUEL_DISCOUNT_PER_REP, FUEL_PRICE_PER_L, MAX_FUEL_DISCOUNT, PLANE_TYPES, REPAIR_DURATION_MS, repRate, WEAR_PER_KM } from '@/lib/game/catalog';
 import { SPECIALS } from '@/lib/game/specials';
-import { getStateSnapshot, parseIcaoQueue } from '@/lib/game/state';
+import { getStateSnapshot, parseIcaoQueue, sseFrame } from '@/lib/game/state';
 import { broadcast } from '@/lib/sse';
 import { IDLE_WORLD_TIME_SCALE } from '@/lib/worldClock';
 import {
@@ -420,12 +420,13 @@ export async function spawnCoveragePackages(
 // --- Public API (singleton DB + lock) — call from every read/action endpoint ---
 
 // ISS-129 (v0.57.0): after each tick on the singleton path, push a fresh snapshot to all SSE
-// subscribers (/api/events) — the exact same body GET /api/state returns (getStateSnapshot), so
-// pushed and polled states can never diverge in shape. Failures are logged, not fatal: polling
-// still works either way. Background ticks (~1 s) cover action mutations for other clients too.
+// subscribers (/api/events). ISS-133 (v0.59.0): frames carry the sseFrame projection — terminal
+// package history is dropped, everything else matches GET /api/state exactly. Failures are logged,
+// not fatal: polling still works either way. Background ticks (~1 s) cover action mutations for
+// other clients too.
 async function broadcastState(): Promise<void> {
   try {
-    broadcast(JSON.stringify(await getStateSnapshot()));
+    broadcast(JSON.stringify(sseFrame(await getStateSnapshot())));
   } catch (err) {
     console.error('flight: state broadcast failed', err);
   }

package.json

@@ -1,6 +1,6 @@
 {
   "name": "flight-ops",
-  "version": "0.58.0",
+  "version": "0.59.0",
   "private": true,
   "scripts": {
     "dev": "next dev -p 3000",