Merge pull request 'Coarsen sub-national regions to a ~100km floor (v0.14.0)' (#28) from feat/coarsen-regions into master
CI / build + unit + e2e (push) Successful in 2m19s
CI / build + unit + e2e (push) Successful in 2m19s
This commit was merged in pull request #28.
This commit is contained in:
@@ -2,6 +2,14 @@
|
|||||||
|
|
||||||
All notable changes to **History** are listed here, newest first.
|
All notable changes to **History** are listed here, newest first.
|
||||||
|
|
||||||
|
## v0.14.0 — 2026-10-12 09:51
|
||||||
|
- **Coarsened the sub-national choropleth dataset to a ~100 km floor (Part of #16).** `public/regions.geo.json` shrinks from 4,138 real administrative regions to **1,796** (2,342 merges): every region smaller than ~100×100 km (10,000 km²) that has a same-country neighbor within 0.2° (≈22 km) of its bounding box is **dissolved into its nearest such neighbor** (nearest by centroid distance; ties → larger area, then id). Standalone cities and micro-districts collapse into the region around them: `US-DC` (191 km²) into Maryland, `FR-75` Paris (59 km²) into Hauts-de-Seine, the UK's 232 admin-1 districts down to **16** regions, and all of North Macedonia's 85 municipalities into one whole-country region.
|
||||||
|
- **Islands are exempt.** A region with no same-country neighbor within 0.2° keeps its shape whatever its size — **83** such small islands survive in the final set (Corsica, the Kerguelen/Crozet archipelagos, the Bahamas districts, Lord Howe, …) and the coarsening's fixed point leaves **0** non-island regions below the floor.
|
||||||
|
- **Real dissolve, not overlap.** Each merge is a real boolean union (`@turf/union`, polygon clipping) that removes the shared boundary between the merged region and its host, yielding one clean polygon — the final dataset has **0** fallback (non-dissolved) unions. Passes iterate smallest-area-first to a fixed point (2 passes on the v0.13 data, ~7 s); the output is sorted by (parent, id) at 4-dp precision and is byte-reproducible: re-running on the coarsened file reaches the fixed point immediately and rewrites an identical file (sha256-verified).
|
||||||
|
- **Numbers.** Per-country region count: min 1 / median 7 / max 80 (Russia; was 1 / 15 / 232 — the UK). Region areas: median 23,933 km²; smallest ≈0 (degenerate micro-dots like Ragged Island and the Paracel Islands, kept as islands); largest is Antarctica itself (≈21.8M km² in the planar model, exempt — it wraps a full 360°). 158 of the 177 countries had at least one merge. The file is 2,431 KB (2.37 MB) — under the 5 MB guard, so no Douglas–Peucker pass was needed.
|
||||||
|
- **Tooling + tests.** New `scripts/coarsen-regions.mjs` (`npm run coarsen:regions`) performs the merge in place; `scripts/regions.test.ts` gains a sixth test asserting that below-floor non-island regions stay under 1% of the dataset (currently 0) alongside the existing well-formed / unique-id / 177-country coverage / area / coverage-band checks (56 tests total). The map's data contract is unchanged (`properties.parent` → RegionMap), so `WorldMap.tsx`, `App.tsx`, `shared/protocol.ts`, and `server/map.ts` all keep working as-is.
|
||||||
|
- Part of #16.
|
||||||
|
|
||||||
## v0.13.0 — 2026-10-12 08:30
|
## v0.13.0 — 2026-10-12 08:30
|
||||||
- **Replaced the geometric sub-region grid with real named sub-national administrative regions (#16, PR 2).** The choropleth source `public/regions.geo.json` now contains real named administrative regions — states, provinces, governorates, oblasts, … (Natural Earth admin-1, 1:10m) — instead of the abstract ~100–200 km geometric grid. Each region keeps a stable `id` and a `name`. The map's data contract is **unchanged** (`properties.parent` → the parent country's ISO_A3, `properties.name` → the region name), so no change to the map code or the LLM contract: `WorldMap.tsx`, `App.tsx`, `shared/protocol.ts`, and `server/map.ts` all keep working as-is.
|
- **Replaced the geometric sub-region grid with real named sub-national administrative regions (#16, PR 2).** The choropleth source `public/regions.geo.json` now contains real named administrative regions — states, provinces, governorates, oblasts, … (Natural Earth admin-1, 1:10m) — instead of the abstract ~100–200 km geometric grid. Each region keeps a stable `id` and a `name`. The map's data contract is **unchanged** (`properties.parent` → the parent country's ISO_A3, `properties.name` → the region name), so no change to the map code or the LLM contract: `WorldMap.tsx`, `App.tsx`, `shared/protocol.ts`, and `server/map.ts` all keep working as-is.
|
||||||
- **Generator rewritten** — `scripts/generate-regions.mjs` (same path, dependency-free, `npm run generate:regions`) now groups the admin-1 features by their `adm0_a3` (the parent's ISO_A3), keeps only parents in the 177-country set, and id's each region with its ISO 3166-2 code when present (e.g. `US-CA`, `FR-75`, `RU-MOW`) else `<ISO_A3>-<nn>`; within a country, ids are disambiguated with a `-2/-3/…` suffix on collision (ISO 3166-2 codes are not always unique within a country — e.g. all nine of Bosnia's entities report `BA-BIH`). A country with no admin-1 data would collapse to one whole-country region `<ISO_A3>-01` (0 are needed at 1:10m, which covers all 177). Coordinates are rounded to 4 decimal places, and the file is Douglas–Peucker-simplified only if it exceeds the ~8 MB guard. Output is sorted by (parent, id) and fully deterministic.
|
- **Generator rewritten** — `scripts/generate-regions.mjs` (same path, dependency-free, `npm run generate:regions`) now groups the admin-1 features by their `adm0_a3` (the parent's ISO_A3), keeps only parents in the 177-country set, and id's each region with its ISO 3166-2 code when present (e.g. `US-CA`, `FR-75`, `RU-MOW`) else `<ISO_A3>-<nn>`; within a country, ids are disambiguated with a `-2/-3/…` suffix on collision (ISO 3166-2 codes are not always unique within a country — e.g. all nine of Bosnia's entities report `BA-BIH`). A country with no admin-1 data would collapse to one whole-country region `<ISO_A3>-01` (0 are needed at 1:10m, which covers all 177). Coordinates are rounded to 4 decimal places, and the file is Douglas–Peucker-simplified only if it exceeds the ~8 MB guard. Output is sorted by (parent, id) and fully deterministic.
|
||||||
|
|||||||
Generated
+77
-2
@@ -1,12 +1,12 @@
|
|||||||
{
|
{
|
||||||
"name": "history",
|
"name": "history",
|
||||||
"version": "0.1.0",
|
"version": "0.13.0",
|
||||||
"lockfileVersion": 3,
|
"lockfileVersion": 3,
|
||||||
"requires": true,
|
"requires": true,
|
||||||
"packages": {
|
"packages": {
|
||||||
"": {
|
"": {
|
||||||
"name": "history",
|
"name": "history",
|
||||||
"version": "0.1.0",
|
"version": "0.13.0",
|
||||||
"dependencies": {
|
"dependencies": {
|
||||||
"@radix-ui/react-dialog": "^1.1.6",
|
"@radix-ui/react-dialog": "^1.1.6",
|
||||||
"@radix-ui/react-label": "^2.1.2",
|
"@radix-ui/react-label": "^2.1.2",
|
||||||
@@ -32,6 +32,7 @@
|
|||||||
"@eslint/js": "^9.20.0",
|
"@eslint/js": "^9.20.0",
|
||||||
"@playwright/test": "^1.62.1",
|
"@playwright/test": "^1.62.1",
|
||||||
"@tailwindcss/vite": "^4.1.4",
|
"@tailwindcss/vite": "^4.1.4",
|
||||||
|
"@turf/union": "^7.4.0",
|
||||||
"@types/better-sqlite3": "^7.6.13",
|
"@types/better-sqlite3": "^7.6.13",
|
||||||
"@types/express": "^4.17.25",
|
"@types/express": "^4.17.25",
|
||||||
"@types/node": "^22.13.0",
|
"@types/node": "^22.13.0",
|
||||||
@@ -3334,6 +3335,52 @@
|
|||||||
"vite": "^5.2.0 || ^6 || ^7 || ^8"
|
"vite": "^5.2.0 || ^6 || ^7 || ^8"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/@turf/helpers": {
|
||||||
|
"version": "7.4.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/@turf/helpers/-/helpers-7.4.0.tgz",
|
||||||
|
"integrity": "sha512-7PAwLZqOdRzTI5g9bHvUwlloAXPDH/mlajtryk0tw4ZwGMtmXsAyF4QundsAMfy4u48Fyd3AUqMXY0SMxqJGWg==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"@types/geojson": "^7946.0.10",
|
||||||
|
"tslib": "^2.8.1"
|
||||||
|
},
|
||||||
|
"funding": {
|
||||||
|
"url": "https://opencollective.com/turf"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/@turf/meta": {
|
||||||
|
"version": "7.4.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/@turf/meta/-/meta-7.4.0.tgz",
|
||||||
|
"integrity": "sha512-3cLUvlEyDuSnMSzrjhaLAEiYR8xhbfWyTVQlrlEw40xL81d4KF4PqUWbjTXKpXZStdYbet2GCurl79KMypNw6g==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"@turf/helpers": "7.4.0",
|
||||||
|
"@types/geojson": "^7946.0.10",
|
||||||
|
"tslib": "^2.8.1"
|
||||||
|
},
|
||||||
|
"funding": {
|
||||||
|
"url": "https://opencollective.com/turf"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"node_modules/@turf/union": {
|
||||||
|
"version": "7.4.0",
|
||||||
|
"resolved": "https://registry.npmjs.org/@turf/union/-/union-7.4.0.tgz",
|
||||||
|
"integrity": "sha512-SYVtsCW9IznTyKlcGWxssRrf90p/hdpch1UeIcZc31pkSVdmeLOyuzPx0ef3UPmaF//fgXAphMph6JWA/FpKbA==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"@turf/helpers": "7.4.0",
|
||||||
|
"@turf/meta": "7.4.0",
|
||||||
|
"@types/geojson": "^7946.0.10",
|
||||||
|
"polyclip-ts": "^0.16.8",
|
||||||
|
"tslib": "^2.8.1"
|
||||||
|
},
|
||||||
|
"funding": {
|
||||||
|
"url": "https://opencollective.com/turf"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/@types/babel__core": {
|
"node_modules/@types/babel__core": {
|
||||||
"version": "7.20.5",
|
"version": "7.20.5",
|
||||||
"resolved": "https://registry.npmjs.org/@types/babel__core/-/babel__core-7.20.5.tgz",
|
"resolved": "https://registry.npmjs.org/@types/babel__core/-/babel__core-7.20.5.tgz",
|
||||||
@@ -4177,6 +4224,16 @@
|
|||||||
"require-from-string": "^2.0.2"
|
"require-from-string": "^2.0.2"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/bignumber.js": {
|
||||||
|
"version": "9.3.1",
|
||||||
|
"resolved": "https://registry.npmjs.org/bignumber.js/-/bignumber.js-9.3.1.tgz",
|
||||||
|
"integrity": "sha512-Ko0uX15oIUS7wJ3Rb30Fs6SkVbLmPBAKdlm7q9+ak9bbIeFf0MwuBsQV6z7+X768/cHsfg+WlysDWJcmthjsjQ==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "MIT",
|
||||||
|
"engines": {
|
||||||
|
"node": "*"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/bindings": {
|
"node_modules/bindings": {
|
||||||
"version": "1.5.0",
|
"version": "1.5.0",
|
||||||
"resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz",
|
"resolved": "https://registry.npmjs.org/bindings/-/bindings-1.5.0.tgz",
|
||||||
@@ -6695,6 +6752,17 @@
|
|||||||
"node": ">=20"
|
"node": ">=20"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/polyclip-ts": {
|
||||||
|
"version": "0.16.8",
|
||||||
|
"resolved": "https://registry.npmjs.org/polyclip-ts/-/polyclip-ts-0.16.8.tgz",
|
||||||
|
"integrity": "sha512-JPtKbDRuPEuAjuTdhR62Gph7Is2BS1Szx69CFOO3g71lpJDFo78k4tFyi+qFOMVPePEzdSKkpGU3NBXPHHjvKQ==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "MIT",
|
||||||
|
"dependencies": {
|
||||||
|
"bignumber.js": "^9.1.0",
|
||||||
|
"splaytree-ts": "^1.0.2"
|
||||||
|
}
|
||||||
|
},
|
||||||
"node_modules/postcss": {
|
"node_modules/postcss": {
|
||||||
"version": "8.5.29",
|
"version": "8.5.29",
|
||||||
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.29.tgz",
|
"resolved": "https://registry.npmjs.org/postcss/-/postcss-8.5.29.tgz",
|
||||||
@@ -7514,6 +7582,13 @@
|
|||||||
"node": ">=0.10.0"
|
"node": ">=0.10.0"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"node_modules/splaytree-ts": {
|
||||||
|
"version": "1.0.2",
|
||||||
|
"resolved": "https://registry.npmjs.org/splaytree-ts/-/splaytree-ts-1.0.2.tgz",
|
||||||
|
"integrity": "sha512-0kGecIZNIReCSiznK3uheYB8sbstLjCZLiwcQwbmLhgHJj2gz6OnSPkVzJQCMnmEz1BQ4gPK59ylhBoEWOhGNA==",
|
||||||
|
"dev": true,
|
||||||
|
"license": "BDS-3-Clause"
|
||||||
|
},
|
||||||
"node_modules/stackback": {
|
"node_modules/stackback": {
|
||||||
"version": "0.0.2",
|
"version": "0.0.2",
|
||||||
"resolved": "https://registry.npmjs.org/stackback/-/stackback-0.0.2.tgz",
|
"resolved": "https://registry.npmjs.org/stackback/-/stackback-0.0.2.tgz",
|
||||||
|
|||||||
+3
-1
@@ -1,6 +1,6 @@
|
|||||||
{
|
{
|
||||||
"name": "history",
|
"name": "history",
|
||||||
"version": "0.13.0",
|
"version": "0.14.0",
|
||||||
"private": true,
|
"private": true,
|
||||||
"type": "module",
|
"type": "module",
|
||||||
"description": "AI-driven alternate-history simulator — the LLM is the world engine (Pax Historia style).",
|
"description": "AI-driven alternate-history simulator — the LLM is the world engine (Pax Historia style).",
|
||||||
@@ -10,6 +10,7 @@
|
|||||||
"start": "node server/dist/index.js",
|
"start": "node server/dist/index.js",
|
||||||
"test": "vitest run",
|
"test": "vitest run",
|
||||||
"generate:regions": "node scripts/generate-regions.mjs",
|
"generate:regions": "node scripts/generate-regions.mjs",
|
||||||
|
"coarsen:regions": "node scripts/coarsen-regions.mjs",
|
||||||
"typecheck": "tsc --noEmit",
|
"typecheck": "tsc --noEmit",
|
||||||
"lint": "eslint .",
|
"lint": "eslint .",
|
||||||
"e2e": "playwright test"
|
"e2e": "playwright test"
|
||||||
@@ -39,6 +40,7 @@
|
|||||||
"@eslint/js": "^9.20.0",
|
"@eslint/js": "^9.20.0",
|
||||||
"@playwright/test": "^1.62.1",
|
"@playwright/test": "^1.62.1",
|
||||||
"@tailwindcss/vite": "^4.1.4",
|
"@tailwindcss/vite": "^4.1.4",
|
||||||
|
"@turf/union": "^7.4.0",
|
||||||
"@types/better-sqlite3": "^7.6.13",
|
"@types/better-sqlite3": "^7.6.13",
|
||||||
"@types/express": "^4.17.25",
|
"@types/express": "^4.17.25",
|
||||||
"@types/node": "^22.13.0",
|
"@types/node": "^22.13.0",
|
||||||
|
|||||||
File diff suppressed because one or more lines are too long
@@ -0,0 +1,612 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
/**
|
||||||
|
* scripts/coarsen-regions.mjs — coarsens the committed `public/regions.geo.json`
|
||||||
|
* in place, so the map is genuinely coarser, not just finer-grained.
|
||||||
|
*
|
||||||
|
* Rule (per user request: "focus on joining cities to the region around it.
|
||||||
|
* Regions shouldn't be smaller than around 100x100km, excluding islands"):
|
||||||
|
*
|
||||||
|
* FLOOR — no region should be smaller than ~100×100 km = 10,000 km²,
|
||||||
|
* EXCEPT islands.
|
||||||
|
* ISLAND — a region is an island if it has NO same-country neighbor: no
|
||||||
|
* other region with the same `properties.parent` whose bounding box
|
||||||
|
* is within CLOSE_DEG (≈≤22 km) of its own. Small islands stay
|
||||||
|
* as-is regardless of size (Corsica, the Hawaiian counties,
|
||||||
|
* Okinawa, the Faroes, island micro-states, …).
|
||||||
|
* MERGE — a region that is small (area < FLOOR) AND not an island is
|
||||||
|
* merged into its NEAREST same-country neighbor (nearest by
|
||||||
|
* centroid distance; tie-break → larger area, then lexicographic
|
||||||
|
* id). Standalone cities (FR-75 Paris, the Latvian/Slovenian
|
||||||
|
* municipalities, the UK's 231 micro-districts) collapse into the
|
||||||
|
* region around them.
|
||||||
|
* DISSOLVE— the merge is a real boolean union (`@turf/union`, polygon-
|
||||||
|
* clipping): the shared boundary between the merged region and its
|
||||||
|
* host is removed, producing a single clean polygon — not two
|
||||||
|
* overlapping shapes. (Fallback if the union fails on a pair:
|
||||||
|
* combine the rings into a MultiPolygon without dissolving; the
|
||||||
|
* count of such fallbacks is reported.)
|
||||||
|
* ITERATE — passes run smallest-area-first (small regions merge into their
|
||||||
|
* neighbors, which grow and then absorb more). After each pass the
|
||||||
|
* area/neighbor/centroid metrics are recomputed; it stops at the
|
||||||
|
* fixed point (no small non-island region left) or at MAX_ITER.
|
||||||
|
* IDENTITY— the host (the region merged INTO) keeps its own `id` and
|
||||||
|
* `name`; the absorbed region's id is dropped. `properties.parent`
|
||||||
|
* is unchanged, so the map code (WorldMap/App/protocol/map.ts)
|
||||||
|
* keeps working untouched.
|
||||||
|
*
|
||||||
|
* Deterministic: fixed processing order (area, then id), deterministic
|
||||||
|
* neighbor choice (distance, then -area, then id), deterministic boolean
|
||||||
|
* union. Re-running on the coarsened file reaches the fixed point
|
||||||
|
* immediately and rewrites a byte-identical file.
|
||||||
|
*
|
||||||
|
* Area model: the SAME planar approximation the validation test uses
|
||||||
|
* (`scripts/regions.test.ts`) — `x = lon·111.32·cos(midLat)`,
|
||||||
|
* `y = lat·110.574` km, shoelace per OUTER ring (holes ignored), each ring at
|
||||||
|
* its own mean latitude — so the script's merge decisions and the test's
|
||||||
|
* assertions measure the same quantity.
|
||||||
|
*
|
||||||
|
* Longitude handling: every ring is unwrapped to a continuous [0,360)-derived
|
||||||
|
* space before its bbox/centroid is computed (the shortest-difference
|
||||||
|
* "np.unwrap" scheme), which makes the one full-360° antimeridian-wrapping
|
||||||
|
* feature (Antarctica's mainland ring) and any seam-straddling ring behave
|
||||||
|
* correctly; cross-feature longitude gaps use the 360°-cyclic minimum.
|
||||||
|
*
|
||||||
|
* Output: features sorted by (parent, id) (the same comparator as
|
||||||
|
* `scripts/generate-regions.mjs`), coordinates rounded to 4 decimal places,
|
||||||
|
* single line of JSON + trailing newline. If the file would exceed the
|
||||||
|
* ~5 MB guard, a light Douglas–Peucker pass (tolerances escalated through a
|
||||||
|
* small table) brings it under ~4 MB before writing.
|
||||||
|
*
|
||||||
|
* Run: `npm run coarsen:regions`
|
||||||
|
*/
|
||||||
|
|
||||||
|
import { readFileSync, writeFileSync } from 'node:fs';
|
||||||
|
import path from 'node:path';
|
||||||
|
import { fileURLToPath } from 'node:url';
|
||||||
|
import { union } from '@turf/union';
|
||||||
|
import { feature, featureCollection } from '@turf/helpers';
|
||||||
|
|
||||||
|
const here = path.dirname(fileURLToPath(import.meta.url));
|
||||||
|
const root = path.resolve(here, '..');
|
||||||
|
const FILE = path.join(root, 'public', 'regions.geo.json');
|
||||||
|
|
||||||
|
// --- parameters ---------------------------------------------------------------
|
||||||
|
const AREA_FLOOR_KM2 = 10_000; // ~100×100 km floor for non-island regions
|
||||||
|
const CLOSE_DEG = 0.2; // "spatially close" bbox gap (both lat and lon ≤ this; ≈≤22 km)
|
||||||
|
const MAX_ITER = 20; // safety cap on fixed-point passes (spec: ~15)
|
||||||
|
const SIZE_GUARD_KB = 5_120; // keep the committed file under ~5 MB
|
||||||
|
const SIZE_TARGET_KB = 4_096; // …and land under ~4 MB once simplified
|
||||||
|
const DP_STEPS = [0.05, 0.1, 0.2, 0.3, 0.5, 0.75, 1.0]; // Douglas–Peucker tolerances (degrees)
|
||||||
|
const R4 = 4; // coordinate precision (decimal places, ≈11 m)
|
||||||
|
const SLIVER_EPS_KM2 = 1e-6; // drop only truly-degenerate (≈zero-area) clipper slivers
|
||||||
|
|
||||||
|
// --- area model (identical to scripts/regions.test.ts) -------------------------
|
||||||
|
const KM_LON = 111.32;
|
||||||
|
const KM_LAT = 110.574;
|
||||||
|
const rad = (deg) => (deg * Math.PI) / 180;
|
||||||
|
|
||||||
|
/** Shoelace area (km²) of one ring, measured at the ring's own mean latitude. */
|
||||||
|
function ringAreaKm2(ring) {
|
||||||
|
let lat = 0;
|
||||||
|
for (const [, la] of ring) lat += la;
|
||||||
|
const kx = KM_LON * Math.cos(rad(ring.length ? lat / ring.length : 0));
|
||||||
|
let a = 0;
|
||||||
|
const n = ring.length;
|
||||||
|
for (let i = 0; i < n; i++) {
|
||||||
|
const [lon1, lat1] = ring[i];
|
||||||
|
const [lon2, lat2] = ring[(i + 1) % n];
|
||||||
|
a += lon1 * kx * (lat2 * KM_LAT) - lon2 * kx * (lat1 * KM_LAT);
|
||||||
|
}
|
||||||
|
return Math.abs(a / 2);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Sum of a geometry's outer-ring areas (holes are lakes — ignored by design). */
|
||||||
|
function geomAreaKm2(geom) {
|
||||||
|
if (!geom) return 0;
|
||||||
|
const polys = geom.type === 'Polygon' ? [geom.coordinates] : geom.coordinates;
|
||||||
|
let total = 0;
|
||||||
|
for (const poly of polys) total += ringAreaKm2(poly[0]);
|
||||||
|
return total;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- longitude unwrapping (antimeridian-safe) ----------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Unwrap a ring's longitudes to a continuous real line (the "np.unwrap"
|
||||||
|
* shortest-difference scheme, anchored at the first vertex's [0,360) value).
|
||||||
|
* Returns { lons, full } — `full` when the unwrapped span is ≥ 300°, i.e. the
|
||||||
|
* ring is a full-360° antimeridian band (Antarctica's mainland ring).
|
||||||
|
*/
|
||||||
|
function unwrapRing(ring) {
|
||||||
|
const n = ring.length;
|
||||||
|
const norm = (x) => ((x % 360) + 360) % 360; // → [0,360)
|
||||||
|
const u = new Array(n);
|
||||||
|
u[0] = norm(ring[0][0]);
|
||||||
|
for (let i = 1; i < n; i++) {
|
||||||
|
// The PREVIOUS unwrapped value may lie outside [0,360) (it accumulates),
|
||||||
|
// so normalize it back before measuring the difference; otherwise a bogus
|
||||||
|
// ~180° jump is inserted and the ring's span balloons to ~180°.
|
||||||
|
const a = norm(u[i - 1]);
|
||||||
|
const b = norm(ring[i][0]);
|
||||||
|
const d = (((b - a) % 360) + 540) % 360 - 180; // shortest signed diff in (-180, 180]
|
||||||
|
u[i] = u[i - 1] + d;
|
||||||
|
}
|
||||||
|
let min = Infinity;
|
||||||
|
let max = -Infinity;
|
||||||
|
for (const v of u) {
|
||||||
|
if (v < min) min = v;
|
||||||
|
if (v > max) max = v;
|
||||||
|
}
|
||||||
|
return { lons: u, full: max - min >= 300 };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Cyclic 1-D gap (degrees) between two tight longitude intervals in their own
|
||||||
|
* unwrapped [0,360)-derived spaces. Tries the three 360° shifts of `b` so
|
||||||
|
* different unwrap anchors reconcile; a `full` interval is close to everything.
|
||||||
|
*/
|
||||||
|
function lonGap(intA, intB) {
|
||||||
|
if (intA.full || intB.full) return 0;
|
||||||
|
let best = Infinity;
|
||||||
|
for (const k of [-1, 0, 1]) {
|
||||||
|
const a2 = intB.a + 360 * k;
|
||||||
|
const b2 = intB.b + 360 * k;
|
||||||
|
const gap = intA.b < a2 ? a2 - intA.b : b2 < intA.a ? intA.a - b2 : 0;
|
||||||
|
if (gap < best) best = gap;
|
||||||
|
}
|
||||||
|
return best;
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- geometry helpers ----------------------------------------------------------
|
||||||
|
const rN = (v, N) => Math.round(v * 10 ** N) / 10 ** N;
|
||||||
|
|
||||||
|
/** Round a ring to 4 dp, drop consecutive duplicates, and (re-)close it. */
|
||||||
|
function roundRing(ring, N = R4) {
|
||||||
|
const out = [];
|
||||||
|
for (const [x, y] of ring) {
|
||||||
|
const p = [rN(x, N), rN(y, N)];
|
||||||
|
const last = out[out.length - 1];
|
||||||
|
if (!last || last[0] !== p[0] || last[1] !== p[1]) out.push(p);
|
||||||
|
}
|
||||||
|
if (
|
||||||
|
out.length > 1 &&
|
||||||
|
out[0][0] === out[out.length - 1][0] &&
|
||||||
|
out[0][1] === out[out.length - 1][1]
|
||||||
|
)
|
||||||
|
out.pop();
|
||||||
|
if (out.length < 3) return out; // degenerate; caller drops it
|
||||||
|
return [...out, [...out[0]]];
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Round every ring of a geometry to 4 dp, drop degenerate rings (<3 pts) and
|
||||||
|
* empty polygons, keep the Polygon/MultiPolygon type (a single-part
|
||||||
|
* MultiPolygon collapses to Polygon). Returns null if nothing usable survived.
|
||||||
|
*/
|
||||||
|
function roundAll(geom) {
|
||||||
|
if (!geom || (geom.type !== 'Polygon' && geom.type !== 'MultiPolygon')) return null;
|
||||||
|
const polys = (geom.type === 'Polygon' ? [geom.coordinates] : geom.coordinates)
|
||||||
|
.map((poly) => poly.map((ring) => roundRing(ring)))
|
||||||
|
.filter((poly) => poly.length >= 1 && poly.every((r) => r.length >= 3));
|
||||||
|
if (!polys.length) return null;
|
||||||
|
if (polys.length === 1 && geom.type === 'Polygon') return { type: 'Polygon', coordinates: polys[0] };
|
||||||
|
return { type: 'MultiPolygon', coordinates: polys };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Normalize a union output (or fallback MultiPolygon): roundAll, then drop
|
||||||
|
* truly-degenerate (≈zero-area, < SLIVER_EPS_KM2) polygons the clipper can
|
||||||
|
* leave behind at shared boundaries. It never drops *every* part — a merge of
|
||||||
|
* two sub-resolution (zero-area) micro-regions must still collapse into one
|
||||||
|
* (degenerate) region rather than vanish. Returns null only when the input
|
||||||
|
* itself had no usable ring (e.g. `null`).
|
||||||
|
*/
|
||||||
|
function normalizeGeom(geom) {
|
||||||
|
const rounded = roundAll(geom);
|
||||||
|
if (!rounded) return null;
|
||||||
|
const all = rounded.type === 'Polygon' ? [rounded.coordinates] : rounded.coordinates;
|
||||||
|
const kept = all.filter((poly) => ringAreaKm2(poly[0]) >= SLIVER_EPS_KM2);
|
||||||
|
const polys = kept.length ? kept : all;
|
||||||
|
if (polys.length === 1 && rounded.type === 'Polygon') return { type: 'Polygon', coordinates: polys[0] };
|
||||||
|
return { type: 'MultiPolygon', coordinates: polys };
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Douglas–Peucker of an OPEN point sequence (keeps endpoints). */
|
||||||
|
function dpOpen(pts, tol) {
|
||||||
|
if (pts.length < 3) return pts;
|
||||||
|
let maxD = 0;
|
||||||
|
let idx = 0;
|
||||||
|
const a = pts[0];
|
||||||
|
const b = pts[pts.length - 1];
|
||||||
|
for (let i = 1; i < pts.length - 1; i++) {
|
||||||
|
const dx = b[0] - a[0];
|
||||||
|
const dy = b[1] - a[1];
|
||||||
|
const l2 = dx * dx + dy * dy;
|
||||||
|
let t = l2 === 0 ? 0 : ((pts[i][0] - a[0]) * dx + (pts[i][1] - a[1]) * dy) / l2;
|
||||||
|
t = Math.max(0, Math.min(1, t));
|
||||||
|
const d = Math.hypot(pts[i][0] - (a[0] + t * dx), pts[i][1] - (a[1] + t * dy));
|
||||||
|
if (d > maxD) {
|
||||||
|
maxD = d;
|
||||||
|
idx = i;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (maxD > tol) {
|
||||||
|
const left = dpOpen(pts.slice(0, idx + 1), tol);
|
||||||
|
const right = dpOpen(pts.slice(idx), tol);
|
||||||
|
return left.concat(right.slice(1));
|
||||||
|
}
|
||||||
|
return [a, b];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Douglas–Peucker a CLOSED ring at tolerance `tol` (0 = skip). */
|
||||||
|
function dpRing(ring, tol) {
|
||||||
|
if (tol <= 0 || ring.length < 4) return ring;
|
||||||
|
let pts = ring;
|
||||||
|
if (pts[0][0] === pts[pts.length - 1][0] && pts[0][1] === pts[pts.length - 1][1]) pts = pts.slice(0, -1);
|
||||||
|
const s = dpOpen(pts, tol);
|
||||||
|
if (s.length < 3) return ring;
|
||||||
|
return [...s, [...s[0]]];
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Round (and optionally DP-simplify) every ring of a geometry. */
|
||||||
|
function transformGeom(geom, tol) {
|
||||||
|
const polys = (geom.type === 'Polygon' ? [geom.coordinates] : geom.coordinates)
|
||||||
|
.map((poly) => poly.map((ring) => roundRing(ring, R4)).map((ring) => (tol > 0 ? dpRing(ring, tol) : ring)))
|
||||||
|
.filter((poly) => poly.length >= 1 && poly.every((r) => r.length >= 3));
|
||||||
|
if (!polys.length) return null;
|
||||||
|
if (polys.length === 1 && geom.type === 'Polygon') return { type: 'Polygon', coordinates: polys[0] };
|
||||||
|
return { type: 'MultiPolygon', coordinates: polys };
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- region record -------------------------------------------------------------
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A region under management. `lonInt` is the tight longitude interval in
|
||||||
|
* [0,360)-derived unwrapped space (or `full` for a 360° band); `(clat, clon)`
|
||||||
|
* is the area-weighted centroid of the LARGEST outer ring (the region's
|
||||||
|
* "location"), unwrapped for its ring then wrapped to [-180,180) for
|
||||||
|
* cyclic-distance use.
|
||||||
|
*/
|
||||||
|
class Region {
|
||||||
|
constructor(f) {
|
||||||
|
this.id = f.id;
|
||||||
|
this.name = f.properties.name;
|
||||||
|
this.parent = f.properties.parent;
|
||||||
|
this.geom = f.geometry;
|
||||||
|
this.rawGeom = null; // last pre-normalization union output (safety net for build())
|
||||||
|
this.dead = false;
|
||||||
|
this.metrics();
|
||||||
|
}
|
||||||
|
|
||||||
|
metrics() {
|
||||||
|
this.area = geomAreaKm2(this.geom);
|
||||||
|
const polys = this.geom.type === 'Polygon' ? [this.geom.coordinates] : this.geom.coordinates;
|
||||||
|
let latMin = Infinity;
|
||||||
|
let latMax = -Infinity;
|
||||||
|
let lonA = Infinity;
|
||||||
|
let lonB = -Infinity;
|
||||||
|
let lonFull = false;
|
||||||
|
let bestRing = null;
|
||||||
|
let bestArea = -1;
|
||||||
|
for (const poly of polys) {
|
||||||
|
const outer = poly[0];
|
||||||
|
for (const [, la] of outer) {
|
||||||
|
if (la < latMin) latMin = la;
|
||||||
|
if (la > latMax) latMax = la;
|
||||||
|
}
|
||||||
|
const { lons, full } = unwrapRing(outer);
|
||||||
|
if (full) lonFull = true;
|
||||||
|
let a = Infinity;
|
||||||
|
let b = -Infinity;
|
||||||
|
for (const v of lons) {
|
||||||
|
if (v < a) a = v;
|
||||||
|
if (v > b) b = v;
|
||||||
|
}
|
||||||
|
lonA = Math.min(lonA, a);
|
||||||
|
lonB = Math.max(lonB, b);
|
||||||
|
const ar = ringAreaKm2(outer);
|
||||||
|
if (ar > bestArea) {
|
||||||
|
bestArea = ar;
|
||||||
|
bestRing = outer;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
this.lonInt = { a: lonA, b: lonB, full: lonFull };
|
||||||
|
this.latMin = latMin;
|
||||||
|
this.latMax = latMax;
|
||||||
|
// area-weighted centroid of the largest outer ring, in unwrapped space
|
||||||
|
const { lons } = unwrapRing(bestRing);
|
||||||
|
const lats = bestRing.map((p) => p[1]);
|
||||||
|
const n = bestRing.length;
|
||||||
|
let A = 0;
|
||||||
|
let cx = 0;
|
||||||
|
let cy = 0;
|
||||||
|
for (let i = 0; i < n; i++) {
|
||||||
|
const x1 = lons[i];
|
||||||
|
const y1 = lats[i];
|
||||||
|
const x2 = lons[(i + 1) % n];
|
||||||
|
const y2 = lats[(i + 1) % n];
|
||||||
|
const f = x1 * y2 - x2 * y1;
|
||||||
|
A += f;
|
||||||
|
cx += (x1 + x2) * f;
|
||||||
|
cy += (y1 + y2) * f;
|
||||||
|
}
|
||||||
|
A /= 2;
|
||||||
|
if (Math.abs(A) > 1e-9) {
|
||||||
|
cx /= 6 * A;
|
||||||
|
cy /= 6 * A;
|
||||||
|
} else {
|
||||||
|
cx = (lonA + lonB) / 2;
|
||||||
|
cy = (latMin + latMax) / 2;
|
||||||
|
}
|
||||||
|
this.clon = (((cx + 180) % 360) + 360) % 360 - 180;
|
||||||
|
this.clat = cy;
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Cyclic planar distance (km) to another region's centroid. */
|
||||||
|
distKm(o) {
|
||||||
|
let dlon = Math.abs(this.clon - o.clon) % 360;
|
||||||
|
if (dlon > 180) dlon = 360 - dlon;
|
||||||
|
const midLat = rad((this.clat + o.clat) / 2);
|
||||||
|
return Math.hypot(dlon * KM_LON * Math.cos(midLat), (this.clat - o.clat) * KM_LAT);
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Bounding-box closeness to another same-space region (see CLOSE_DEG). */
|
||||||
|
closeTo(o) {
|
||||||
|
const latGap = this.latMax < o.latMin ? o.latMin - this.latMax : o.latMax < this.latMin ? this.latMin - o.latMax : 0;
|
||||||
|
if (latGap > CLOSE_DEG) return false;
|
||||||
|
return lonGap(this.lonInt, o.lonInt) <= CLOSE_DEG;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A "degenerate" region is a zero-area artifact: after 4-dp rounding its only
|
||||||
|
* ring has <3 distinct points (a line/point, e.g. a micro-municipality that
|
||||||
|
* v0.13's Douglas–Peucker collapsed to `[A, B, A]`). It contributes no area,
|
||||||
|
* so it can never be dissolved by the clipper (which returns null) nor kept
|
||||||
|
* through ring-dropping (which loses the 2-point ring). Merging one is purely
|
||||||
|
* an absorption: the host keeps its own geometry.
|
||||||
|
*/
|
||||||
|
const isDegenerate = (r) => r.area < SLIVER_EPS_KM2;
|
||||||
|
|
||||||
|
// --- main ----------------------------------------------------------------------
|
||||||
|
function main() {
|
||||||
|
const fc0 = JSON.parse(readFileSync(FILE, 'utf8'));
|
||||||
|
const regions = fc0.features.map((f) => new Region(f));
|
||||||
|
const byParent = new Map();
|
||||||
|
for (const r of regions) {
|
||||||
|
if (!byParent.has(r.parent)) byParent.set(r.parent, []);
|
||||||
|
byParent.get(r.parent).push(r);
|
||||||
|
}
|
||||||
|
|
||||||
|
const t0 = Date.now();
|
||||||
|
console.log(`input: ${fc0.features.length} regions across ${byParent.size} countries (${(Buffer.byteLength(readFileSync(FILE)) / 1024).toFixed(0)} KB)`);
|
||||||
|
console.log(`parameters: floor ${AREA_FLOOR_KM2.toLocaleString('en-US')} km2, close ${CLOSE_DEG} deg, max ${MAX_ITER} passes`);
|
||||||
|
|
||||||
|
const merges = []; // { from, fromName, fromArea, to, toName, toAreaAfter }
|
||||||
|
let fallbackUnions = 0;
|
||||||
|
|
||||||
|
// The fixed-point loop. Each pass: every region that is small AND has a
|
||||||
|
// close same-country neighbor merges into the nearest such neighbor
|
||||||
|
// (nearest by centroid distance; tie → larger area → lexicographic id).
|
||||||
|
for (let pass = 0; pass < MAX_ITER; pass++) {
|
||||||
|
let skippedIsland = 0;
|
||||||
|
let skippedGrew = 0;
|
||||||
|
const items = [];
|
||||||
|
for (const r of regions) {
|
||||||
|
if (r.dead || r.area >= AREA_FLOOR_KM2) continue;
|
||||||
|
const siblings = byParent.get(r.parent);
|
||||||
|
for (const s of siblings) {
|
||||||
|
if (s !== r && !s.dead && r.closeTo(s)) {
|
||||||
|
items.push(r);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (!items.length) {
|
||||||
|
console.log(`after pass ${pass}: fixed point reached (no small non-island regions left)`);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
items.sort((a, b) => (a.area !== b.area ? a.area - b.area : a.id < b.id ? -1 : 1));
|
||||||
|
let merged = 0;
|
||||||
|
for (const r of items) {
|
||||||
|
if (r.dead) continue;
|
||||||
|
if (r.area >= AREA_FLOOR_KM2) {
|
||||||
|
skippedGrew++;
|
||||||
|
continue; // absorbed a smaller region earlier this pass and crossed the floor
|
||||||
|
}
|
||||||
|
const siblings = byParent.get(r.parent);
|
||||||
|
// Close same-country neighbors as of NOW (mid-pass): some may have died
|
||||||
|
// earlier in this pass.
|
||||||
|
const close = [];
|
||||||
|
for (const s of siblings) if (s !== r && !s.dead && r.closeTo(s)) close.push(s);
|
||||||
|
if (!close.length) {
|
||||||
|
skippedIsland++;
|
||||||
|
continue; // lost its last close neighbor this pass → island now → exempt
|
||||||
|
}
|
||||||
|
// Prefer a REAL (non-degenerate) host so real regions dissolve cleanly;
|
||||||
|
// fall back to any close neighbor only when no real one is nearby.
|
||||||
|
const realCands = close.filter((s) => !isDegenerate(s));
|
||||||
|
const pool = realCands.length ? realCands : close;
|
||||||
|
let host = null;
|
||||||
|
for (const s of pool) {
|
||||||
|
if (
|
||||||
|
!host ||
|
||||||
|
r.distKm(s) < r.distKm(host) ||
|
||||||
|
(r.distKm(s) === r.distKm(host) &&
|
||||||
|
(s.area !== host.area ? s.area > host.area : s.id < host.id))
|
||||||
|
)
|
||||||
|
host = s;
|
||||||
|
}
|
||||||
|
if (isDegenerate(r)) {
|
||||||
|
// r is a zero-area artifact (line/point): it contributes no area, so the
|
||||||
|
// merge is a pure absorption — the host keeps its own geometry (a real
|
||||||
|
// host is untouched; if the host is also degenerate a cluster of dots
|
||||||
|
// collapses to one). Never route degenerate input through the clipper
|
||||||
|
// (returns null) or ring-dropping (would lose the 2-point ring the map
|
||||||
|
// already renders).
|
||||||
|
r.dead = true;
|
||||||
|
merged++;
|
||||||
|
merges.push({ from: r.id, fromName: r.name, fromArea: r.area, to: host.id, toName: host.name, toAreaAfter: host.area });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if (isDegenerate(host)) {
|
||||||
|
// r is real but the chosen host is a zero-area artifact: the host takes
|
||||||
|
// on r's geometry (keeping its own id/name). No clipper needed.
|
||||||
|
host.geom = r.geom;
|
||||||
|
host.rawGeom = r.geom;
|
||||||
|
host.metrics();
|
||||||
|
r.dead = true;
|
||||||
|
merged++;
|
||||||
|
merges.push({ from: r.id, fromName: r.name, fromArea: r.area, to: host.id, toName: host.name, toAreaAfter: host.area });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
// Both real: dissolve via a boolean union; the host keeps its id/name
|
||||||
|
// and the item vanishes. @turf/union@7 dispatches through geomEach, which
|
||||||
|
// only understands a FeatureCollection / single Feature / bare geometry —
|
||||||
|
// a raw array of features has no `.type` and throws "Unknown Geometry
|
||||||
|
// Type". Wrap it.
|
||||||
|
let unionGeom = null;
|
||||||
|
let rawUnion = null;
|
||||||
|
try {
|
||||||
|
const u = union(featureCollection([feature(r.geom), feature(host.geom)]));
|
||||||
|
rawUnion = u ? u.geometry : null;
|
||||||
|
unionGeom = normalizeGeom(rawUnion);
|
||||||
|
} catch (err) {
|
||||||
|
console.error(` [warn] union failed for ${r.id} into ${host.id} (${err.message}); combining without dissolve`);
|
||||||
|
fallbackUnions++;
|
||||||
|
}
|
||||||
|
if (!unionGeom) {
|
||||||
|
// fallback: no dissolve — keep both shapes in one MultiPolygon
|
||||||
|
const polys = [...(r.geom.type === 'Polygon' ? [r.geom.coordinates] : r.geom.coordinates), ...(host.geom.type === 'Polygon' ? [host.geom.coordinates] : host.geom.coordinates)];
|
||||||
|
rawUnion = { type: 'MultiPolygon', coordinates: polys };
|
||||||
|
unionGeom = normalizeGeom(rawUnion);
|
||||||
|
if (!unionGeom) fallbackUnions++;
|
||||||
|
}
|
||||||
|
if (!unionGeom) {
|
||||||
|
console.error(` [error] could not merge ${r.id} into ${host.id}; leaving ${r.id} as-is`);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
host.geom = unionGeom;
|
||||||
|
host.rawGeom = rawUnion;
|
||||||
|
host.metrics();
|
||||||
|
r.dead = true;
|
||||||
|
merged++;
|
||||||
|
merges.push({ from: r.id, fromName: r.name, fromArea: r.area, to: host.id, toName: host.name, toAreaAfter: host.area });
|
||||||
|
}
|
||||||
|
const alive = regions.filter((r) => !r.dead).length;
|
||||||
|
console.log(
|
||||||
|
`pass ${pass + 1}: ${items.length} small non-island region(s) → ${merged} merged` +
|
||||||
|
` (${skippedGrew} crossed the floor, ${skippedIsland} became island) → ${alive} regions remain`,
|
||||||
|
);
|
||||||
|
if (merged === 0) {
|
||||||
|
// items exist but nothing could merge (all became islands this pass) → stable
|
||||||
|
console.log(`pass ${pass + 1}: no merges possible — stable state`);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// --- assemble output ----------------------------------------------------------
|
||||||
|
const alive = regions.filter((r) => !r.dead);
|
||||||
|
const sortById = (a, b) =>
|
||||||
|
a.parent !== b.parent ? (a.parent < b.parent ? -1 : 1) : a.id.localeCompare(b.id, 'en', { numeric: true });
|
||||||
|
alive.sort(sortById);
|
||||||
|
|
||||||
|
const build = (tol) => {
|
||||||
|
const feats = [];
|
||||||
|
for (const r of alive) {
|
||||||
|
// Never drop a region: if the simplified/normalized geometry degenerated,
|
||||||
|
// fall back to the raw (pre-normalization) union output, then to the
|
||||||
|
// untouched input geometry.
|
||||||
|
const g =
|
||||||
|
transformGeom(r.geom, tol) ??
|
||||||
|
(r.rawGeom ? transformGeom(r.rawGeom, 0) ?? r.rawGeom : r.geom);
|
||||||
|
feats.push({ type: 'Feature', id: r.id, properties: { name: r.name, parent: r.parent }, geometry: g });
|
||||||
|
}
|
||||||
|
return { type: 'FeatureCollection', features: feats };
|
||||||
|
};
|
||||||
|
|
||||||
|
let out = build(0);
|
||||||
|
let json = JSON.stringify(out);
|
||||||
|
let sizeKB = Buffer.byteLength(json) / 1024;
|
||||||
|
console.log(`\ncoarsened: ${fc0.features.length} → ${out.features.length} regions (${fc0.features.length - out.features.length} merged away)`);
|
||||||
|
|
||||||
|
let usedTol = 0;
|
||||||
|
if (sizeKB > SIZE_GUARD_KB) {
|
||||||
|
for (const t of DP_STEPS) {
|
||||||
|
out = build(t);
|
||||||
|
json = JSON.stringify(out);
|
||||||
|
sizeKB = Buffer.byteLength(json) / 1024;
|
||||||
|
usedTol = t;
|
||||||
|
console.log(`DP ${t}: ${sizeKB.toFixed(1)} KB`);
|
||||||
|
if (sizeKB < SIZE_TARGET_KB) break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
writeFileSync(FILE, json + '\n');
|
||||||
|
|
||||||
|
// --- summary -------------------------------------------------------------------
|
||||||
|
const areaOf = new Map();
|
||||||
|
const counts = new Map();
|
||||||
|
let maxA = 0;
|
||||||
|
let maxR = null;
|
||||||
|
for (const f of out.features) {
|
||||||
|
const a = geomAreaKm2(f.geometry);
|
||||||
|
areaOf.set(f.id, a);
|
||||||
|
counts.set(f.properties.parent, (counts.get(f.properties.parent) ?? 0) + 1);
|
||||||
|
if (a > maxA) {
|
||||||
|
maxA = a;
|
||||||
|
maxR = f;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
const ccounts = [...counts.values()].sort((a, b) => a - b);
|
||||||
|
const median =
|
||||||
|
ccounts.length % 2 ? ccounts[(ccounts.length - 1) / 2] : (ccounts[ccounts.length / 2 - 1] + ccounts[ccounts.length / 2]) / 2;
|
||||||
|
const maxCountry = [...counts.entries()].sort((a, b) => b[1] - a[1])[0];
|
||||||
|
|
||||||
|
// islands = small regions with no close same-country neighbor in the FINAL set
|
||||||
|
const finalByParent = new Map();
|
||||||
|
for (const f of out.features) {
|
||||||
|
if (!finalByParent.has(f.properties.parent)) finalByParent.set(f.properties.parent, []);
|
||||||
|
finalByParent.get(f.properties.parent).push(f);
|
||||||
|
}
|
||||||
|
// rebuild final metrics (light: only need island flag on the < FLOOR set)
|
||||||
|
const finalRegions = out.features.map((f) => new Region(f));
|
||||||
|
const finalByParent2 = new Map();
|
||||||
|
for (const r of finalRegions) {
|
||||||
|
if (!finalByParent2.has(r.parent)) finalByParent2.set(r.parent, []);
|
||||||
|
finalByParent2.get(r.parent).push(r);
|
||||||
|
}
|
||||||
|
let smallIslands = 0;
|
||||||
|
let smallNonIslands = 0;
|
||||||
|
const islandSamples = [];
|
||||||
|
for (const r of finalRegions) {
|
||||||
|
if (r.area >= AREA_FLOOR_KM2) continue;
|
||||||
|
let hasNeighbor = false;
|
||||||
|
for (const s of finalByParent2.get(r.parent)) {
|
||||||
|
if (s !== r && r.closeTo(s)) {
|
||||||
|
hasNeighbor = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (hasNeighbor) smallNonIslands++;
|
||||||
|
else {
|
||||||
|
smallIslands++;
|
||||||
|
if (islandSamples.length < 12) islandSamples.push(`${r.id} ${r.name} (${Math.round(r.area)} km2)`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`wrote ${FILE} — ${sizeKB.toFixed(1)} KB (${(sizeKB / 1024).toFixed(2)} MB)` + (usedTol ? ` [DP @ ${usedTol}°]` : ''));
|
||||||
|
console.log(`per-country count: min ${ccounts[0]} / median ${median} / max ${ccounts[ccounts.length - 1]} (max: ${maxCountry[0]})`);
|
||||||
|
console.log(`max region: ${maxR.id} (${maxR.properties.name}) ≈ ${Math.round(maxA).toLocaleString('en-US')} km2`);
|
||||||
|
console.log(`small (<${AREA_FLOOR_KM2.toLocaleString('en-US')} km2) in final set: ${smallIslands + smallNonIslands} — ${smallIslands} islands (exempt) / ${smallNonIslands} non-island residuals`);
|
||||||
|
if (islandSamples.length) console.log(`island samples: ${islandSamples.join(', ')}`);
|
||||||
|
console.log(`fallback (non-dissolved) unions: ${fallbackUnions}`);
|
||||||
|
console.log(`merge log (${merges.length} merges): first 15:`);
|
||||||
|
for (const m of merges.slice(0, 15)) {
|
||||||
|
console.log(` ${m.from} (${m.fromName}, ${Math.round(m.fromArea)} km2) → ${m.to} (${m.toName}, now ${Math.round(m.toAreaAfter)} km2)`);
|
||||||
|
}
|
||||||
|
const logPath = '/tmp/opencode/coarsen-log.txt';
|
||||||
|
writeFileSync(logPath, merges.map((m) => `${m.from}\t${m.fromName}\t${Math.round(m.fromArea)}\t${m.to}\t${m.toName}\t${Math.round(m.toAreaAfter)}`).join('\n') + '\n');
|
||||||
|
console.log(`full merge log → ${logPath}; elapsed ${((Date.now() - t0) / 1000).toFixed(1)}s`);
|
||||||
|
}
|
||||||
|
|
||||||
|
main();
|
||||||
+144
-6
@@ -1,12 +1,16 @@
|
|||||||
/**
|
/**
|
||||||
* scripts/regions.test.ts — validation for the committed `public/regions.geo.json`.
|
* scripts/regions.test.ts — validation for the committed `public/regions.geo.json`.
|
||||||
*
|
*
|
||||||
* The dataset is now REAL named sub-national administrative regions (Natural
|
* The dataset is REAL named sub-national administrative regions (Natural
|
||||||
* Earth admin-1: states, provinces, governorates, oblasts, …) rather than the
|
* Earth admin-1: states, provinces, governorates, oblasts, …) that the
|
||||||
* old geometric grid — one feature per named region, each carrying
|
* coarsening step (`scripts/coarsen-regions.mjs`, v0.14) has merged down to a
|
||||||
* `properties = { name, parent }` where `parent` is the country's ISO_A3 (the
|
* ~100×100 km (10,000 km²) floor: every small non-island region was dissolved
|
||||||
* RegionMap key the map resolves ownership through). Tiny or uncovered
|
* into its nearest same-country neighbor, so the only regions left below the
|
||||||
* countries collapse to a single whole-country region (`<ISO_A3>-01`).
|
* floor are islands (no same-country sibling nearby). One feature per named
|
||||||
|
* region, each carrying `properties = { name, parent }` where `parent` is the
|
||||||
|
* country's ISO_A3 (the RegionMap key the map resolves ownership through).
|
||||||
|
* Tiny or uncovered countries collapse to a single whole-country region
|
||||||
|
* (`<ISO_A3>-01`).
|
||||||
*
|
*
|
||||||
* This test is self-contained (no import of the generator): it re-implements
|
* This test is self-contained (no import of the generator): it re-implements
|
||||||
* the same planar area model the generator/analysis use, so the area-based
|
* the same planar area model the generator/analysis use, so the area-based
|
||||||
@@ -109,6 +113,99 @@ const countByParent = new Map<string, number>();
|
|||||||
for (const f of regions.features)
|
for (const f of regions.features)
|
||||||
countByParent.set(f.properties.parent, (countByParent.get(f.properties.parent) ?? 0) + 1);
|
countByParent.set(f.properties.parent, (countByParent.get(f.properties.parent) ?? 0) + 1);
|
||||||
|
|
||||||
|
// --- island/close-neighbor helpers (mirror scripts/coarsen-regions.mjs) ------
|
||||||
|
|
||||||
|
interface LonInt {
|
||||||
|
a: number;
|
||||||
|
b: number;
|
||||||
|
full: boolean;
|
||||||
|
}
|
||||||
|
interface RegionBBox {
|
||||||
|
latMin: number;
|
||||||
|
latMax: number;
|
||||||
|
lon: LonInt;
|
||||||
|
}
|
||||||
|
|
||||||
|
const COARSE_FLOOR_KM2 = 10_000; // the coarsener's ~100×100 km floor
|
||||||
|
const CLOSE_DEG = 0.2; // "spatially close" bbox gap (both lat and lon ≤ this)
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Unwrap a ring's longitudes to a continuous line (np.unwrap shortest-diff
|
||||||
|
* scheme, anchored at the first vertex's [0,360) value). `full` when the
|
||||||
|
* unwrapped span is ≥ 300° (a full-360° antimeridian band, e.g. Antarctica).
|
||||||
|
* Must stay in sync with the coarsening script.
|
||||||
|
*/
|
||||||
|
function unwrapRing(ring: Ring) {
|
||||||
|
const n = ring.length;
|
||||||
|
const norm = (x: number) => ((x % 360) + 360) % 360;
|
||||||
|
const u: number[] = new Array(n);
|
||||||
|
u[0] = norm(ring[0][0]);
|
||||||
|
for (let i = 1; i < n; i++) {
|
||||||
|
// the previous UNWRAPPED value may lie outside [0,360); normalize it
|
||||||
|
// before measuring the shortest signed difference, or a bogus ~180° jump
|
||||||
|
// is inserted and the ring's span balloons.
|
||||||
|
const a = norm(u[i - 1]);
|
||||||
|
const b = norm(ring[i][0]);
|
||||||
|
const d = (((b - a) % 360) + 540) % 360 - 180;
|
||||||
|
u[i] = u[i - 1] + d;
|
||||||
|
}
|
||||||
|
let min = Infinity;
|
||||||
|
let max = -Infinity;
|
||||||
|
for (const v of u) {
|
||||||
|
if (v < min) min = v;
|
||||||
|
if (v > max) max = v;
|
||||||
|
}
|
||||||
|
return { lons: u, full: max - min >= 300 };
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Cyclic 1-D gap (degrees) between two tight longitude intervals in their own
|
||||||
|
* unwrapped spaces; a `full` interval is close to everything.
|
||||||
|
*/
|
||||||
|
function lonGap(intA: LonInt, intB: LonInt): number {
|
||||||
|
if (intA.full || intB.full) return 0;
|
||||||
|
let best = Infinity;
|
||||||
|
for (const k of [-1, 0, 1]) {
|
||||||
|
const a2 = intB.a + 360 * k;
|
||||||
|
const b2 = intB.b + 360 * k;
|
||||||
|
const gap = intA.b < a2 ? a2 - intA.b : b2 < intA.a ? intA.a - b2 : 0;
|
||||||
|
if (gap < best) best = gap;
|
||||||
|
}
|
||||||
|
return best;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Per-region bbox in antimeridian-safe unwrapped longitude space.
|
||||||
|
const regionBBox = new Map<string, RegionBBox>();
|
||||||
|
for (const f of regions.features) {
|
||||||
|
const polys = f.geometry.type === 'Polygon' ? [f.geometry.coordinates] : f.geometry.coordinates;
|
||||||
|
let latMin = Infinity;
|
||||||
|
let latMax = -Infinity;
|
||||||
|
let a = Infinity;
|
||||||
|
let b = -Infinity;
|
||||||
|
let full = false;
|
||||||
|
for (const poly of polys) {
|
||||||
|
const outer = poly[0];
|
||||||
|
for (const [, la] of outer) {
|
||||||
|
if (la < latMin) latMin = la;
|
||||||
|
if (la > latMax) latMax = la;
|
||||||
|
}
|
||||||
|
const { lons, full: fl } = unwrapRing(outer);
|
||||||
|
if (fl) full = true;
|
||||||
|
for (const v of lons) {
|
||||||
|
if (v < a) a = v;
|
||||||
|
if (v > b) b = v;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
regionBBox.set(f.id, { latMin, latMax, lon: { a, b, full } });
|
||||||
|
}
|
||||||
|
|
||||||
|
/** Bounding-box closeness (same definition the coarsening script uses). */
|
||||||
|
function bboxesClose(x: RegionBBox, y: RegionBBox): boolean {
|
||||||
|
const latGap = x.latMax < y.latMin ? y.latMin - x.latMax : y.latMax < x.latMin ? x.latMin - y.latMax : 0;
|
||||||
|
if (latGap > CLOSE_DEG) return false;
|
||||||
|
return lonGap(x.lon, y.lon) <= CLOSE_DEG;
|
||||||
|
}
|
||||||
|
|
||||||
describe('regions.geo.json', () => {
|
describe('regions.geo.json', () => {
|
||||||
it('is a valid GeoJSON FeatureCollection of well-formed features', () => {
|
it('is a valid GeoJSON FeatureCollection of well-formed features', () => {
|
||||||
expect(regions.type).toBe('FeatureCollection');
|
expect(regions.type).toBe('FeatureCollection');
|
||||||
@@ -188,6 +285,47 @@ describe('regions.geo.json', () => {
|
|||||||
);
|
);
|
||||||
});
|
});
|
||||||
|
|
||||||
|
it('leaves only islands below the ~10,000 km2 coarsening floor (non-island small regions merged)', () => {
|
||||||
|
// v0.14 coarsening: every region smaller than the floor that has a
|
||||||
|
// same-country sibling within CLOSE_DEG of its bbox is dissolved into its
|
||||||
|
// nearest such neighbor, so a below-floor region may only survive if it
|
||||||
|
// is an island. Count the non-island residuals below the floor and allow
|
||||||
|
// a small tolerance band (strictly < 1% of all regions) — the coarsener's
|
||||||
|
// fixed point leaves none, and this catches a regression in the merge
|
||||||
|
// logic (e.g. a broken neighbor test that exempts land regions) without
|
||||||
|
// being brittle to one or two borderline cases.
|
||||||
|
const smallNonIslands: string[] = [];
|
||||||
|
let smallIslands = 0;
|
||||||
|
let smallTotal = 0;
|
||||||
|
for (const f of regions.features) {
|
||||||
|
const a = regionArea.get(f.id) ?? 0;
|
||||||
|
if (a >= COARSE_FLOOR_KM2) continue;
|
||||||
|
smallTotal++;
|
||||||
|
const bb = regionBBox.get(f.id)!;
|
||||||
|
let hasNeighbor = false;
|
||||||
|
for (const g of regions.features) {
|
||||||
|
if (g.id === f.id || g.properties.parent !== f.properties.parent) continue;
|
||||||
|
if (bboxesClose(bb, regionBBox.get(g.id)!)) {
|
||||||
|
hasNeighbor = true;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (hasNeighbor) {
|
||||||
|
smallNonIslands.push(`${f.id} (${Math.round(a)} km2)`);
|
||||||
|
} else smallIslands++;
|
||||||
|
}
|
||||||
|
const frac = regions.features.length ? smallNonIslands.length / regions.features.length : 0;
|
||||||
|
expect(
|
||||||
|
frac < 0.01,
|
||||||
|
`${smallNonIslands.length} non-island region(s) below the ${COARSE_FLOOR_KM2} km2 floor ` +
|
||||||
|
`(${(100 * frac).toFixed(2)}% of all regions): ${smallNonIslands.slice(0, 10).join(', ')}`,
|
||||||
|
).toBe(true);
|
||||||
|
console.log(
|
||||||
|
`\n below-floor regions: ${smallTotal} — ${smallIslands} islands (exempt, no same-country neighbor within ${CLOSE_DEG}°)` +
|
||||||
|
` / ${smallNonIslands.length} non-island residuals (must stay < 1% of all regions)`,
|
||||||
|
);
|
||||||
|
});
|
||||||
|
|
||||||
it('sums to within [40%, 175%] of each country area (median in [70%, 130%])', () => {
|
it('sums to within [40%, 175%] of each country area (median in [70%, 130%])', () => {
|
||||||
// The band is a planar-approximation sanity check. It is wider than a naive
|
// The band is a planar-approximation sanity check. It is wider than a naive
|
||||||
// [40%, 110%] because Natural Earth's admin-1 polygons are digitized
|
// [40%, 110%] because Natural Earth's admin-1 polygons are digitized
|
||||||
|
|||||||
Reference in New Issue
Block a user