#Developing GeoSpaX

Building from source, the test and benchmark scripts, the engine API, and the measurements and findings behind the design. For using GeoSpaX, see the user guide.

#Status

It grew out of a Phase 0 spike that covered the riskiest part of a Rust + Tauri ArcGIS-Pro-class GIS: moving lots of geodata from a native Rust engine into a web-tech UI, with one code path for the desktop and web editions.

• Phase 0 built the data plane.
• Phase 1, slice 1 adds:
  • GDAL's columnar Arrow reader;
  • overview pyramids (a background job that writes PMTiles, with density heatmaps at low zoom);
  • a Symbology pane (single symbol, unique values, graduated colours with natural breaks / quantile / equal interval);
  • legends, draw order and a persisted project.

                 ┌──────────── web/ (TypeScript, MapLibre GL JS 6, deck.gl 9.4) ────────────┐
                 │ Automate (recipes, Auto, runs) · Contents · Symbology · Devices · map    │
                 │  Backend interface:  HttpBackend  |  TauriBackend (scheme | invoke)      │
                 └───────────────┬───────────────────────────────────┬──────────────────────┘
                     HTTP (gzip MVT, raw Arrow)        gis:// custom scheme · invoke → raw bytes
          ┌──────── crates/server (Axum, 180 lines) ──┐   ┌── apps/desktop (Tauri 2.12, 114 lines) ─┐
          └──────────────────┬────────────────────────┘   └──────────────────┬──────────────────────┘
                             └────────────── Engine::handle(ApiRequest) ─────┘
          ┌──────────────────────── crates/engine (Rust, ~15.7k lines) ─────────────────────────┐
          │ reader:   GDAL Arrow stream (RFC 86) | per-feature, ignored-fields column pruning   │
          │ tiles:    WKB → project → Douglas-Peucker → clip → quantise → MVT (hand-rolled pbf) │
          │ overview: z0–12 pyramid job → PMTiles (+ density points), served transparently      │
          │ jobs:     background pool, progress, cancel · classify: Jenks/quantile/equal, unique│
          │ catalog · pooled GDAL handles · LRU · Arrow IPC export · project file · stats       │
          │ devices:  phones/cameras/drones/GPS → photos, tracks, waypoints · live GNSS          │
          │ sensors:  logs, FIT, HTTP, MQTT(S), LoRaWAN, SensorThings → readings · triggers      │
          │ rasters:  DEM/terrain/STAC indices · GRASS/SAGA providers                            │
          │ automation: recipes · runner · triggers · AutoMap · layout → PNG/SVG · report ·     │
          │           SQL/buffer/hexbin/IDW · R & Python sidecars · CLI (run/check/watch)        │
          └──────────────────────────────────────────────────────────────────────────────────────┘
                  GeoPackage · FlatGeobuf · Shapefile · FileGDB · … (GDAL ≥ 3.6; 3.12+ recommended)

#What it proves

Full numbers: bench/RESULTS.md. Screenshots: bench/screenshots/.

#Findings that change the plan

1. Keep two storage roles. Use GeoPackage as the editable workspace and FlatGeobuf as the read-optimized render cache. GeoPackage spends about 30% of each tile in GDAL's per-query setup. FlatGeobuf's packed Hilbert R-tree spends about 0%.

2. On-the-fly tiling needs an overview pyramid below about z13 for dense layers. Done in Phase 1, see finding 12. Live tiling now serves only z13+ (and, later, edits). The engine still auto-raises the live minzoom when an average tile would exceed 10k features.

3. GDAL reading is the bottleneck, not our code. Our whole pipeline (WKB parse → projection → simplification → clipping → encoding) costs about 1 µs per feature. Per-feature GDAL calls were 50–80% of tile time, and 99% of the Arrow export. Phase 1 switched to the Arrow stream (finding 10). With FlatGeobuf, our own processing is now about half of tile time, so it is the next optimization target.

4. Don't gzip Arrow. Float64 coordinates barely compress: 13% smaller but 80% slower, as measured. The server now gzips MVT and JSON only.

5. deck.gl 9.4's interleaved MapLibre integration (MapboxOverlay({interleaved: true})) breaks on MapLibre 6. It divides by map.transform.height, and v6 removed map.transform. Overlaid mode also touches it when terrain is on. GeoSpaX runs a standalone Deck synced through MapLibre's public camera API instead (web/src/main.ts). Revisit when deck.gl ships v6 support.

6. The MapLibre 6 worker works with Vite 8 via import workerUrl from 'maplibre-gl/dist/maplibre-gl-worker.mjs?worker&url' plus worker.format: 'es'.

7. Desktop transports are close. Custom scheme vs invoke + raw bytes gave 926 vs 793 ms mean settle under software GL, which is within noise. Measure on real GPUs before choosing. Keep both behind the Backend interface.

8. Pool GDAL handles. Opening a handle ranges from about 20 ms (GeoPackage) to about 0.9 s (Shapefile, which loads its .qix index). The engine opens each handle once per worker and reuses it.

9. Rust and GDAL threading. gdal::Dataset is Send, but CoordTransform isn't. The engine therefore uses per-worker pooled handles (dataset + transforms) on a bounded rayon pool, and async adapters never run GDAL on an executor thread.

10. Read through GDAL's Arrow stream by default (--reader arrow; features remains as a fallback). GDAL emits batches of 16k rows (OGR_L_GetArrowStream), we tell it which columns to skip (OGR_L_SetIgnoredFields), and arrow-rs reads them zero-copy over the C Data Interface. Mean polygon-tile time drops 19–44% at the same output (108/108 tiles byte-identical); point tiles barely change. The 1M-point Arrow export drops 4.4×.

11. GDAL < 3.12 has a race in the GeoPackage Arrow path that crashes multi-batch bbox reads. We found it while building overviews: random segfaults inside GDAL's SQLite worker thread, or in our code reading corrupted batches. In 3.10's GetNextArrowArrayAsynchronous, the consumer waits for nCountRows == 0 && !bIsFinished, so it can take a batch the worker thread is still writing. GDAL 3.12.0 replaced this with an explicit bThreadReady handshake (we diffed 3.10.3 / 3.11.3 / 3.12.0). The engine checks the runtime GDAL version. On < 3.12 it routes bbox GPKG reads through GDAL's generic Arrow implementation (a thread-local OGR_GPKG_STREAM_BASE_IMPL). That path is still 20–30% faster than per-feature reads. Whole-table reads (export, column scans) keep the fast multi-threaded path. Ship desktop builds with GDAL ≥ 3.12 (bundled-gdal or conda-forge); 16-client stress test: 255/255 requests, 0 errors. Related: a spatial filter that covers the whole layer is now dropped, because GDAL then swaps the R-tree for an ST_EnvIntersects() SQL filter that crashed in the same worker.

12. Overview pyramids work best with density points, not feature dropping. The job renders z0–N with the normal tile pipeline. Features smaller than 2 px are not dropped but counted on a 4 px grid, giving a <layer>:density point layer with a count attribute. Low-zoom tiles stay small (z0–5: 0.1–9 KB) and the map still shows where the data is. The engine serves z ≤ N from the PMTiles file and renders z > N live, so URLs don't change. Each archive is keyed by a fingerprint of the source (path, size, mtime, feature count), and a stale archive is ignored. Builds run on a separate job pool with progress and cancel. Open issue: the densest z12 tile is 840 KB, because many buildings sit just over 2 px; a stricter threshold or MLT would shrink it.

13. Calibrate density heatmaps from data, not constants. My hand-tuned weights were 10× off: saturated at first, invisible after one fix. The build job now records, per zoom, the 90th percentile of each tile's densest cell (z9: 166 features; z12: 4) in a sidecar JSON. The frontend sets weight = sqrt(count / p90[z]) per integer zoom (step, because tiles are per zoom), with intensity set from the kernel's cell overlap.

14. Encode only the attributes the symbology needs. Tiles carry a small default attribute set; the UI requests more with ?fields=height,kind (part of the cache key). Classification runs in the engine on one column (geometry ignored), so the browser never downloads 1M rows to compute breaks. The DP is optimal 1-D k-means (Ckmeans.1d.dp) with divide-and-conquer, which is what "Jenks natural breaks" approximates. It is unit-tested against brute force.

15. Persist the project as one JSON document now; move to CIM later. GET/PUT /api/project stores layer styles, draw order and the view (atomic write to <data>/.geospax/project.json). The schema is deliberately small; mapping it onto CIM (CIMUniqueValueRenderer, CIMClassBreaksRenderer) is a Phase 2 item.

16. Give control-plane requests their own lane. scripts/bench-api-stress.py showed that job polling, project saves and stats waited about 500 ms (p50) behind tile renders and column scans, because every request went through the 2-thread tile pool. That would stall a progress bar while the user pans. Engine::dispatch now routes data-plane requests (tiles, Arrow, classify, unique) to the tile pool and everything else to a small control pool: p50 0.5 ms, p95 ≤ 2 ms under the same 16-client load. Both adapters (Axum and the Tauri gis:// protocol) use the same function.

17. WebKitGTK (the Linux desktop webview) silently drops whole tiles with the default WebGL drawing buffer. Under software GL (Xvfb + llvmpipe), 25 of 29 desktop sessions lost at least one (source, tile) pair at random positions. This happened on both transports, with or without the heatmap, and with compositing disabled. A probe (?gldebug=1) showed MapLibre holding every tile loaded, renderable and with geometry, and gl.getError() was clean: the draws are lost inside the webview. preserveDrawingBuffer: true or MSAA fixes it (17 / 17 sessions complete). GeoSpaX turns preserveDrawingBuffer on for WebKitGTK only, because it costs a frame copy, more on tile-based GPUs. Override it with ?glpdb=0|1 or ?glaa=1. Still to check: GPU-backed WebKitGTK, and a minimal MapLibre-only reproduction to report upstream. Tauri 3's CEF runtime is the escape hatch if WebKitGTK stays fragile.

18. Real-world EXIF is often slightly broken, so parse it tolerantly. Photos written by GDAL (and the HEIC made from them) were rejected wholesale by the strict EXIF parser ("Unexpected next IFD"), although the GPS fields were intact. With continue_on_error, every sample reads: 12/12 phone JPEGs, 15/15 drone JPEGs with DJI XMP gimbal data, a HEIC, and two genuine Nikon photos from exif-samples. Lazy users never learn why a photo "has no location", so the reader must not be the reason.

19. Runtime layers need a registry, not a fixed list. Device imports add layers while the app runs. The engine's layer list became an append-only RwLock registry with stable indexes (the tile cache keys on them). Re-imports replace layers in place and clear the cache; imports persist across restarts.

20. Per-layer style rules are not enough; the map needs map-level rules. The first automatic maps followed every per-layer rule and still went wrong:

    • an orange coverage polygon hid orange photo points;
    • red sample dots disappeared into a red ramp;
    • a white mesh covered an interpolated surface.

    AutoMap now also decides at map level:

    • single colours stay distinct within one map;
    • points over a coloured surface turn white with a dark ring;
    • one to three areas over other data are see-through;
    • dense polygon layers drop outlines;
    • the step that made a layer can hint how it should look (IDW: no outlines; hexbins: outlines).

    Each rule adds its reason to the run log.

21. Never call out while holding a lock you might need again. The first server run with device recipes on Auto froze. The device hook started a run while holding the active-runs lock, and run creation looked up runs under the same lock: a self-deadlock. CLI runs never hit it because they don't take that lock. Run ids are now unique by their folder alone, and lock order is documented in the code.

22. Write attributes by the field's real index, not its position. KML adds Name and Description fields of its own, and Shapefile truncates names to 10 characters. Writing by position therefore shifted values: a timestamp ended up in length_m, and GDAL warned. The writer now records each field's index after it is created.

23. Validation must list every problem at once. The test suite caught that a structural error (a dependency cycle) hid per-step errors (an unknown step name, a missing parameter). Lazy users fix everything in one pass, so the validator now collects all of them. Unknown step names suggest the nearest real one.

24. Kriging beat cross-validated IDW by 28 % RMSE on the same samples (0.106 vs 0.147). Picking the method automatically is worth it; that is the planned interpolate.auto. Almost all of the 4.2 s R step is interpreter start-up and package loading, so a warm R session is the obvious next optimisation. Debian 13 does not package gstat; install it from CRAN on top of Debian's r-cran-sf.

25. Don't trust STAC raster metadata blindly. Earth Search marks Sentinel-2 L2A items earthsearch:boa_offset_applied: true yet still lists offset: -0.1 in raster:bands. Applying the offset again pushed red reflectance to about zero and NDVI to a mean of 0.90 with the 98th percentile at 1.00. With the flag honoured, the mean is 0.55 (0.13–0.85), which fits Mt Hagen's gardens, grassland and forest. The test suite now fails if NDVI sticks near 1.

26. "Clear on average" is not "clear over my field". Scene cloud cover is a tile-wide number, so rs.index reads the 20 m scene classification for just the area first (one small COG read). It skips dates that are under 70 % clear there before fetching any bands.

27. Read windows, not scenes. COG range reads make remote data cheap enough for automation. The 11 × 9 km area needs about 1.3 MB of a 3600 × 3600 Copernicus tile (2.7–3.8 s including reprojection), and a 13 × 13 km Sentinel-2 band window takes 2.1 s, versus 100+ MB per full band.

28. One coordinate helper per meaning. The composer's own merc() returns normalised tile space; GDAL wants metres. Using the wrong one rendered every raster for a 0.0002 m box at 0°, 0°: transparent maps with correct legends. Rasters are now warped with an explicitly named metre-based function.

29. Reuse the GIS engines people trust instead of re-implementing them. GRASS r.geomorphon (ten landforms) and SAGA's wetness index each took one step of glue. Inputs and outputs move as GeoTIFF, and GRASS's own category labels and colours become the legend. Wrapping them took less code than re-implementing either algorithm.

30. When a standard exists, classes should carry its meaning. On the air-quality walk, quantile breaks for PM2.5 came out at 5.5 / 8 / 9 / 10.1 / 15.7 / 61.1 µg/m³. That painted 80 % of a clean walk orange and red and hid the one roadside hotspot. Anchoring classes at the WHO 2021 guideline and interim targets (15 / 25 / 37.5 / 50 / 75) gives a green walk with one hotspot running yellow to red. Maps from different days now compare, and the legend says "guideline met" instead of a number nobody can act on.

31. Sensor files need no format registry; the headers are enough. Latitude, longitude and time columns are recognised by name, and units are taken from brackets (PM2.5 (µg/m³)). That read every sample without configuration: logger CSV, semicolon CSV with decimal commas, JSON lines with nested locations or GeoJSON geometries, and FIT. Rows logged before the GPS had a fix (empty or 0/0) are counted and skipped instead of being plotted off the coast of Africa.

32. A device's kind must follow what it sends, not whether it sends numbers. The first version turned an OwnTracks phone into a "sensor" because the phone reports its battery level; the MQTT test caught it. A minimal MQTT 3.1.1 subscriber on a plain thread is about 200 lines (CONNECT, SUBSCRIBE, PUBLISH, PINGREQ, QoS-1 acks, back-off), so the engine needs no async MQTT stack. It is tested end to end against Mosquitto 2.0.21.

33. The first map export in a fresh process can take 3 s on a cold disk. fontdb scans every system font before the first render. The same export takes 0.1 s once the OS file cache is warm. Desktop builds should bundle a small font set and load only that, both for speed and for maps that look the same on every machine.

34. TLS fits the thread-per-source MQTT client without going async. rustls' StreamOwned wraps the same TcpStream, read timeouts stay on the socket underneath, and rustls keeps partial records across a timeout, so the 1-second poll loop did not change. Errors are worth translating: "invalid peer certificate: UnknownIssuer" became "not signed by a CA this computer trusts; for a private broker add ?ca=", which is the fix. For SensorThings, the poller pushes a station only when its newest observation time moves, because the live store keeps every push as a reading and triggers would otherwise fire on every poll.

35. With LiDAR, the data's own classes beat any filter, but only the right ones. The Autzen survey's stadium stands are left unclassified, so "surface minus ground minus buildings" turned them into 30 m of canopy with tree tops on the seats, and a transmission tower became a 58 m tree. Building the canopy from the vegetation classes (3–5) fixed both. Water returns (class 9) have to count as bare earth or rivers become holes in the DTM. Headers are written before coordinates are rounded to the file's scale, so a grid sized exactly to them drops edge points. And contours traced on a 50 cm DTM turned centimetre noise into 485 tiny loops; on a 2 m average there are 12 clean lines.

36. A 3D view needs its own framing and colour rules, not the 2D map's. The first rendered scenes looked wrong in ways no unit test caught. The point cloud, drawn over the model, speckled the terrain and painted every roof in the building class's orange. Autzen's DTM ends in no-data around the irregular survey edge, so the grid-border walls vanished and the terrain floated. White sample dots, the right 2D rule for points over a coloured raster, lay on the 3D terrain like snow. Fitting the camera to the bounding box's corners left the model filling only 20 % of the window, because those corners are mostly empty air. The fixes: the picture shows points only when there is no model; walls follow the edge of the data; light dots are recoloured for 3D; the camera is fitted to actual vertices and re-centred. Baking the hillshade's sun into every colour and marking materials unlit kept the scene identical in Blender, the browser and the CPU renderer. The Khronos validator found nothing to fix, because 16-bit indices, 4-byte alignment and min/max bounds were designed in from the start.

37. Matching R digit for digit is not the same as being understood. GeoSpaX's Gi*, Moran's I, variograms and kriging equal R's spdep and gstat to 10⁻⁶ or better, yet three results would still have misled a lazy reader. The pH variogram's best fit (gstat's too) has a 38 km range on a 4 km study area. That is mathematically right, but the samples follow a trend and the variogram never levels off, so the report now says "trend" instead of quoting a range. On a random field, the fitted nugget was a quarter of the true one (gstat's fit did the same), so the standard errors were twice too small (RMSSE 2.0). The report now flags any RMSSE outside 0.67–1.5 and says how to read the uncertainty map. Hexagon centres put dozens of neighbour pairs exactly at the automatic distance band, where a rounding error would decide whether they count; a one-in-a-billion slack makes the answer deterministic. The cartography needed its own fixes. "Not significant" cells vanished into the land colour. The frame-filling layout that suits DEMs cut the edges off surfaces clipped to the samples' hull.

#Run it

#Prerequisites

On Debian 12+/Ubuntu 22.04+, scripts/setup-linux.sh [--test-tools] [--r] [--rea] [--windows] [--swap 4G] installs everything below (system packages, Rust, Node 24, and optionally the headless-test tools, R with sf + gstat for the R recipe steps, REA, and the Windows cross-build tools). If a copy of the repo lost the executable bits, run chmod +x scripts/*.sh scripts/*.py (or call them through bash/python3).

• Rust via rustup (tested with 1.99) and Node.js 22+.
• GDAL ≥ 3.6 with headers (Arrow stream API). Use 3.12+ for the fast GeoPackage bbox path (finding 11).
  • Debian/Ubuntu: sudo apt install libgdal-dev gdal-bin pkg-config clang
  • macOS: brew install gdal pkg-config
  • Windows: install GDAL (OSGeo4W or conda), then set GDAL_HOME (containing include/ and lib/) and GDAL_VERSION (e.g. 3.10.3), and put GDAL's bin on PATH. Alternatively, build GDAL from source with --features geospax-engine/bundled-gdal (needs CMake and a C++ toolchain; the first build is slow).
  • Windows binaries from Linux: scripts/setup-linux.sh --windows && scripts/package-windows.sh (MinGW-w64 + conda-forge GDAL; see RESULTS §15).
• Tauri 2 prerequisites:
  • Windows: WebView2, already present on 10/11.
  • macOS: Xcode Command Line Tools.
  • Linux: libwebkit2gtk-4.1-dev libgtk-3-dev libayatana-appindicator3-dev librsvg2-dev libsoup-3.0-dev libxdo-dev libssl-dev.

#1. Data (≈800 MB: Natural Earth + 1M synthetic buildings and points)

scripts/data-fetch-samples.sh data          # bash; on Windows use Git Bash or WSL

Natural Earth is public domain. The "buildings" are synthetic: random rectangles clustered around five Papua New Guinea highland towns.

#2. Web edition

(cd web && npm install && npm run build)
cargo run --release -p geospax-server -- --data data   # → http://localhost:8080 (this computer only; see SECURITY.md)

For frontend development, run npm --prefix web run dev (port 5173, which proxies /api and /tiles to :8080). The developer stats panel (FPS, tile timings, cache) is hidden from 1.0 on: add ?hud=1 to the URL to show it (benchmark runs show it automatically).

Server flags:

• --reader arrow|features
• --state-dir DIR: overviews and project; default <data>/.geospax, or GEOSPAX_STATE for desktop
• --threads N, --no-cache, --no-gzip, --bench-log FILE
• Automation: --recipes DIR (extra recipe folder), --auto id,id|all (triggers on), --no-triggers, --inbox DIR (drop folder: the matching device recipe maps whatever lands there, no Auto switch needed); subcommands run, check, watch, recipes, steps
• Installed (.deb, container): without --web the server serves /usr/share/geospax/web

Environment:

• GEOSPAX_GPKG_FILTERED=async|base|features overrides the GDAL-version-based choice.
• GEOSPAX_VALIDATE_ARROW=1 runs validate_full() on every Arrow batch from GDAL (debugging).

#Try the device hub (plug in, get a map)

python3 scripts/data-sample-devices.py /tmp/geospax-devices      # sample phone, drone card, GPS and inbox (synthetic)
./target/release/geospax-server --data data \
  --watch /tmp/geospax-devices/TEST-PHONE --watch /tmp/geospax-devices/DJI-SD \
  --watch /tmp/geospax-devices/GARMIN --watch /tmp/geospax-devices/inbox \
  --gps "file:///tmp/geospax-devices/live-replay.nmea?rate=1"    # add --auto-import to map new devices automatically

• Real devices need no flags. Phones and cards mounted under /media/$USER, /run/media/$USER, gvfs MTP/PTP mounts, /Volumes (macOS), or removable drives and camera/GPS cards on Windows drive letters are found automatically.
• Live GNSS: --gps /dev/ttyUSB0, serial:///dev/ttyACM0?baud=9600, tcp://host:port or gpsd://localhost:2947.
• Desktop edition: the same options through GEOSPAX_WATCH, GEOSPAX_AUTO_IMPORT=1 and GEOSPAX_GPS.
• In the UI, Devices lists what was found. Import & map adds photos, tracks, waypoints and sensor readings as layers, styles them with AutoMap (photo wedges, track casing, readings coloured by the main measurement) and zooms to them; they persist across restarts.
• Send from phone: add --phone (or press Devices → Send from phone → Turn on) and scan the QR code with a phone on the same Wi-Fi: it sends photos, tracks and logs, takes geotagged photos and shares live location, and the map comes back to the phone (DEVICES.md §3a).
• Sensor networks: --mqtt "mqtts://APP@ttn:API-KEY@eu1.cloud.thethings.network/v3/%2B/devices/%2B/up" maps every LoRaWAN sensor of a The Things Stack application; ChirpStack works the same way. --sta https://host/v1.1 turns the stations of an OGC SensorThings service (public air-quality and water networks) into live sensors (DEVICES.md §4).

#Try automation (maps make themselves)

python3 scripts/data-sample-devices.py /tmp/geospax-devices     # sample phone, drone card, GPS, inbox
python3 scripts/data-sample-field.py data                        # 60 soil samples for the geostatistics recipes
python3 scripts/data-sample-incidents.py data                    # 1,120 incidents in Lae with 3 planted clusters (hot spots)

# One command, one finished map + report (PNG, SVG, report.html in <state>/runs/<run-id>/)
./target/release/geospax-server run photos-to-map --data data --input source=/tmp/geospax-devices/TEST-PHONE
./target/release/geospax-server run points-to-hexbins --data data          # 1M points -> hexagons in ~0.7 s
./target/release/geospax-server run interpolate-samples --data data        # IDW, power chosen by cross-validation
./target/release/geospax-server check recipes/drone-flight-report.yaml     # validate + show the plan
./target/release/geospax-server recipes | steps                            # what is installed / available

# Plug in and forget: device recipes on Auto (a phone -> photos-to-map, a drone card -> drone-flight-report, …)
./target/release/geospax-server --data data --watch /tmp/geospax-devices/DJI-SD --watch /tmp/geospax-devices/GARMIN \
  --auto photos-to-map,track-to-map,drone-flight-report

• UI: the Automate pane (bottom dock).

  • Run a recipe and watch its steps.
  • Open the finished Map and Report.
  • Switch on Auto to let triggers run it.
  • Save this map as a recipe to turn the current map into a re-runnable recipe.

• Desktop: the desktop binary takes the same commands (geospax run …, geospax watch). Environment variables: GEOSPAX_AUTO=id,id|all, GEOSPAX_RECIPES=dir.

• R and Python steps need Rscript (with sf + gstat for r-kriging) or python3 on PATH, or GEOSPAX_RSCRIPT / GEOSPAX_PYTHON. On Debian/Ubuntu:

  apt install r-base-core r-base-dev r-cran-sf r-cran-sp r-cran-zoo r-cran-fnn r-cran-spacetime r-cran-sftime
  Rscript -e 'install.packages("gstat")'      # or simply: scripts/setup-linux.sh --r
  

• Your own recipes: put them in <state>/recipes/ or a --recipes DIR folder (format: AUTOMATION §2).

#Try sensors (loggers, watches, live feeds)

python3 scripts/data-sample-devices.py /tmp/geospax-devices     # includes AQ-LOGGER/ (air-quality CSV) and WATCH/ (FIT run)
./target/release/geospax-server run sensor-log-to-map --data data --input source=/tmp/geospax-devices/AQ-LOGGER
./target/release/geospax-server run track-to-map --data data --input source=/tmp/geospax-devices/WATCH   # + heart rate

# Live sensors + an alert recipe: an MQTT broker (apt install mosquitto), the server subscribed, a simulated feed
mosquitto -p 1883 &
./target/release/geospax-server --data data --web web/dist --mqtt "mqtt://127.0.0.1:1883/sensors/#" --auto air-quality-alert &
python3 scripts/demo-sensor-feed.py --mqtt 127.0.0.1:1883        # or --url http://127.0.0.1:8080 to push over HTTP
curl -X POST localhost:8080/api/devices/live -d '{"id":"aq-01","lat":-5.86,"lon":144.23,"values":{"pm2_5":48}}'

# Sensor networks: a broker over TLS with your own CA, LoRaWAN from The Things Stack, an OGC SensorThings service
./target/release/geospax-server --data data --web web/dist --auto air-quality-alert \
  --mqtt "mqtts://broker.example.org/sensors/%23?ca=/etc/geospax/ca.pem" \
  --mqtt "mqtts://my-app@ttn:NNSXS.KEY@eu1.cloud.thethings.network/v3/%2B/devices/%2B/up" \
  --sta "https://sensorthings.example.org/FROST-Server/v1.1?every=300"

• Plug in a logger. Any folder or card whose CSV/TSV/JSON files have latitude/longitude columns is a sensor device. Turn on sensor-log-to-map Auto and plugging it in gives the map. Phone sensor-app exports work the same way.
• Phones without an app. OwnTracks publishing to your broker shows up as a live phone.
• Sensor networks. LoRaWAN uplinks (The Things Stack, ChirpStack) and OGC SensorThings stations are live sensors too, so air-quality-alert fires when any of them reports PM2.5 above 35.
• Alerts. air-quality-alert fires when any live sensor reports PM2.5 above 35 µg/m³, at most every 30 minutes per sensor. Copy it to watch another field or level. sensor: { inside: <polygon layer> } fires when a phone or sensor enters an area (DEVICES §4).

#Try terrain, satellite and GRASS/SAGA recipes (needs internet)

./target/release/geospax-server run terrain-report --data data          # Copernicus DEM -> relief + contours (PNG/SVG/PDF), slope classes
./target/release/geospax-server run latest-ndvi --data data             # newest clear Sentinel-2 pass -> NDVI, classes, true colour
./target/release/geospax-server run landforms-grass --data data         # GRASS r.geomorphon (apt install grass-core)
./target/release/geospax-server run wetness-saga --data data            # SAGA wetness index (apt install saga)
./target/release/geospax-server run terrain-report --data data --input area=my_parcels   # any layer as the area

• The area is any layer (the default is the most sample-like one). Only that window of the DEM or imagery is read. Rasters appear in the Rasters panel and the maps.
• latest-ndvi carries a stac trigger: switch on Auto and it re-runs when a newer clear pass appears over the area.
• Add - webhook: { token: <16+ chars> } to any recipe to start it with POST /api/hooks/<token>. The JSON body becomes its inputs.

#Try LiDAR (point clouds → terrain, canopy, trees, buildings)

python3 scripts/data-sample-lidar.py /tmp/scene.las                      # synthetic survey near Lae with known truth (add --classified)
./target/release/geospax-server run lidar-to-terrain --input source=/tmp/scene.las
./target/release/geospax-server run lidar-to-terrain --input source=/path/to/tiles --input resolution=0.5
./target/release/geospax-server run lidar-to-terrain --input source=file.laz --input epsg=32755   # file without a coordinate system

• Ground comes from the file's classes when it has them, else the SMRF filter finds it. Outputs: dtm, dsm and chm GeoTIFFs, trees and buildings layers, a terrain map, a canopy map and a report.
• With the web edition started with --inbox DIR (or the desktop's ~/GeoSpaX/Inbox), dropping a folder of LAS/LAZ files there runs it by itself; a drive with point clouds shows up as a lidar device.

#Try 3D scenes (terrain, buildings, trees, point clouds)

./target/release/geospax-server run lidar-to-terrain --input source=/tmp/scene.las   # → scene.glb, scene.html, scene.png in the run folder
./target/release/geospax-server run terrain-report --data data                      # terrain block diagram with contours (needs internet)
xdg-open "$(ls -d data/.geospax/runs/*terrain-report | tail -1)/scene.html"           # the offline viewer

• scene.glb is glTF 2.0 (Blender, Windows 3D Viewer, ArcGIS Pro, CesiumJS); scene.html is a self-contained WebGL viewer with the model inside; the report shows the rendered 3D picture. In the app, the run card's 🧊 3D button opens it.
• Any recipe can add - { id: scene, use: scene.3d, with: { dem: dem } } after a DEM (AUTOMATION §6d).

#Try the Phase 1 features

1. Open ⚙ (or the layer name) on synthetic.buildings. Choose Graduated colors → field height → Natural breaks (or Quantile / Equal interval) → 3–10 classes → a colour ramp. Or choose Unique values → kind. Class colours are editable; the legend appears under the layer in Contents.
2. In the same pane, click Build overviews (z0–12) and watch the progress bar (about 30 s). Then zoom out to z9: a density heatmap. Polygons take over from z13.
3. Reorder layers with ▲▼, and change opacity or visible zoom range. Everything persists across reloads (?fresh ignores the saved project).

#3. Desktop edition

(cd apps/desktop && npm install)
GEOSPAX_DATA="$PWD/data" npm --prefix apps/desktop run dev                       # dev: Vite + Tauri
cargo build --release -p geospax --features geospax/custom-protocol && ./target/release/geospax data
(cd apps/desktop && npx tauri build)                                             # installers (MSI/NSIS, DMG, deb/rpm/AppImage)

On Linux with NVIDIA, if the window is blank, set WEBKIT_DISABLE_DMABUF_RENDERER=1 (see Tauri's Linux graphics notes). On Linux the app enables WebGL preserveDrawingBuffer automatically, to work around WebKitGTK dropping tiles (finding 17).

#4. Measure on your machines (the Phase 0 exit criterion)

# Engine latency, headless, per format and zoom:
cargo run --release -p geospax-engine --bin geospax-bench -- --data data \
  --layer synthetic.buildings --layer synthetic_buildings.fgb --zooms 11,12,13,14 --arrow synthetic.points \
  --reader arrow            # or: --reader features
# Overview pyramid build time, size and serving latency (writes to --state-dir):
cargo run --release -p geospax-engine --bin geospax-bench -- --data data --overview synthetic.buildings --state-dir /tmp/ov

In the app, click Run benchmark (a 9-stop camera tour recording FPS and settle time), then Download JSON. Do this on:

On Windows, the quickest way to fill the first row is the Windows installer (scripts/package-windows.sh). Install it, start GeoSpaX and click Run benchmark.

#Scripts

Names follow setup-*, data-*, demo-*, test-*, bench-* (test-web-* for browser tests). scripts/test-all.sh DATA [OUT] runs everything with one command and prints a pass/fail summary. Headless runs (CI, or a server without a GPU) exercise the same paths, but their FPS numbers are meaningless.

#Layout

docs/            USER_GUIDE.md · VISION.md · FEATURE_MATRIX.md · AUTOMATION.md · DEVICES.md
CHANGELOG.md · LICENSE (MIT)
Dockerfile · docker-compose.yml    web edition image (+ MQTT broker); .github/workflows/ci.yml: CI and releases
releases/        packages built by scripts/package-linux.sh (.deb) and scripts/package-windows.sh (setup.exe)
packaging/windows/  Tauri NSIS overlay, installer hooks, geospax-web.cmd, README.txt for the install folder
recipes/         gallery recipes (YAML), embedded in the binaries: photos-to-map · track-to-map ·
                 drone-flight-report · points-to-hexbins · points-to-hotspots · interpolate-samples · python-spatial-stats · r-kriging ·
                 terrain-report · lidar-to-terrain · latest-ndvi · landforms-grass · wetness-saga · sensor-log-to-map ·
                 air-quality-alert
crates/engine/   reader.rs (Arrow | features) · catalog.rs · wkb.rs · tile.rs · mvt.rs (+ density grid)
                 overview.rs · pmtiles_io.rs (sync PMTiles v3 reader) · jobs.rs · classify.rs
                 devices.rs (device hub: discovery, EXIF/XMP + sidecars, GPX/NMEA/SRT, photos placed by time,
                             import, live store, sensor hook) · phone.rs (phone link: HTTPS, QR, uploads, batches)
                 sensors.rs (sensor CSV/JSON/FIT readers, live HTTP/MQTT messages, LoRaWAN uplinks (TTN, ChirpStack),
                             MQTT 3.1.1 client with TLS (rustls, Mozilla roots, own CA, client certificates), WHO levels)
                 sta.rs (OGC SensorThings polling: newest observation per datastream, nextLink paging)
                 raster.rs (catalogue, stats, auto-style, Web Mercator overlays, GDAL warp/translate/DEM/HTTP)
                 lidar.rs (LAS/LAZ headers and streaming, grids, SMRF ground, DSM, canopy, buildings, tree tops)
                 geostats.rs (neighbour index, Moran's I, Gi*, FDR, distance search, variograms, kriging, CV)
                 automation/ recipe.rs (parse, validate, ${…}) · mod.rs (registry, runner, runs) · steps.rs
                             automap.rs · cartography.rs (layout → SVG, resvg PNG, krilla PDF) · report.rs · features.rs
                             raster_steps.rs (DEM, terrain, contours, classify, sample, STAC + indices)
                             lidar_steps.rs (lidar.grid: rasters, tree and building layers, report story)
                             providers.rs (GRASS, SAGA) · sidecar.rs (R/Python)
                             triggers.rs (folder, schedule, stac) · cli.rs (run/check/watch)
                 router.rs · arrow_export.rs · bin/geospax-bench.rs
crates/server/   Axum adapter, binary `geospax-server` (static files + /api + /tiles; gzip except Arrow)
apps/desktop/    Tauri 2 adapter (gis:// scheme, `tile` command, GEOSPAX_* env hooks); binary `geospax`
web/             MapLibre 6 + deck.gl frontend shared by both editions
                 main.ts · automate.ts (Automate panel) · rasters.ts (Rasters panel) · symbology.ts · panel.ts
                 devices.ts · phone.ts (Send from phone card) · backend.ts · public/phone.html (phone page)
scripts/         setup-* · data-* · demo-* · test-* · bench-*  (see Scripts)
bench/           RESULTS.md · engine-*.json · reader-*.json · overviews.json · classify.json · stress.json
                 desktop-tile-drops.json · ui-*-headless.json · screenshots/

#API (identical over HTTP and gis://)

#Not in scope (yet)

• MLT encoding. Martin can transcode MVT to MLT. It would shrink the dense z12 pyramid tiles.
• Bottom-up pyramid builds. The builder re-reads the source at every zoom; z6–12 take 3–4.5 s each. Aggregating density counts from the zoom below would cut build time for point-like data.
• Symbol editing beyond colour, CIM import/export. (Automatic point labels exist in exported maps; interactive labels don't yet.)
• Raster tiles (rasters are served as one overlay image each; tiles matter for very large rasters).
• CEF on Linux. Waiting for Tauri 3 to stabilize.
• Web-map export, for_each, the step cache, OGC SensorThings, cloud-free composites, land-cover classification, more providers (Whitebox, OTB, SNAP, ODM), the optional AI recipe writer. Specified in docs/AUTOMATION.md; Phase 2 onwards.