14 Server
The server is for serving up any web services outside those of Github (e.g., website, docs and R package msens) using Docker (see the docker-compose.yml; with reverse proxying from subdomains to ports by Caddy).
The full architecture is summarized in the Software overview. This chapter documents the concrete services running on the host and how Caddy maps subdomains to container ports.
14.1 Setup
For the latest instructions on launching an Amazon instance and installing the server software, see Server Setup · MarineSensitivity/server Wiki, which is pasted below for convenience…
on AWS as EC2 instance using Docker
14.1.1 launch instance
name: msens1:
- Software Image (AMI)
Canonical, Ubuntu, 22.04 LTS, amd64 jammy image build on 2023-09-19 ami-0fc5d935ebf8bc3bc - Virtual server type (instance type)
t2.xlarge (4 vCPU, 16 GB memory)
- Firewall (security group)
New security group - Storage (volumes)
2 volume(s)- 20 GB
/server software, disposable - 60 GB
/sharefor all data, persistent and to be backed up
- 20 GB
14.1.1.1 allocate IP address
Elastic IP addresses | EC2 | us-east-1 for persistent IP address
Allocated IPv4 address:
100.25.173.0Associate Elastic IP address
14.1.2 ssh to server
pem='/Users/bbest/My Drive/private/msens_key_pair.pem'
ssh -i $pem ubuntu@msens1.marinesensitivity.org14.1.2.1 set hostname
sudo vi /etc/cloud/cloud.cfg
# preserve_hostname: true
sudo hostnamectl set-hostname msens1.marinesensitivity.org
sudo reboot14.1.2.2 mount volume
The extra volume (60 GB for /share) was added during EC2 launch instance wizard, but needs to be mounted before available for use.
df -HFilesystem Size Used Avail Use% Mounted on
/dev/root 21G 2.3G 19G 11% /
tmpfs 8.4G 0 8.4G 0% /dev/shm
tmpfs 3.4G 898k 3.4G 1% /run
tmpfs 5.3M 0 5.3M 0% /run/lock
/dev/xvda15 110M 6.4M 104M 6% /boot/efi
tmpfs 1.7G 4.1k 1.7G 1% /run/user/1000lsblkNAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
loop0 7:0 0 24.6M 1 loop /snap/amazon-ssm-agent/7528
loop1 7:1 0 55.7M 1 loop /snap/core18/2790
loop2 7:2 0 63.5M 1 loop /snap/core20/2015
loop3 7:3 0 111.9M 1 loop /snap/lxd/24322
loop4 7:4 0 40.8M 1 loop /snap/snapd/20092
xvda 202:0 0 20G 0 disk
├─xvda1 202:1 0 19.9G 0 part /
├─xvda14 202:14 0 4M 0 part
└─xvda15 202:15 0 106M 0 part /boot/efi
xvdb 202:16 0 60G 0 disksudo file -s /dev/xvdb
# /dev/xvdb: dataSo no file system on /dev/xvdb yet.
sudo mkfs -t xfs /dev/xvdb
sudo mkdir /share
sudo mount /dev/xvdb /sharesudo cp /etc/fstab /etc/fstab.orig
sudo blkid
# /dev/xvdb: UUID="bc766dfb-1c42-49cf-9320-2242a2d48a2e" BLOCK_SIZE="512" TYPE="xfs"
sudo vim /etc/fstab
# UUID=bc766dfb-1c42-49cf-9320-2242a2d48a2e /share xfs defaults,nofail 0 2
df -h
sudo umount /share ; df -h
sudo mount -a ; df -h14.1.3 install docker
Following:
sudo apt-get update
#OLD: sudo apt-get install docker.io -yNEW: [[Migrate to docker compose]]
sudo systemctl start docker
sudo docker run hello-world
sudo systemctl enable docker
docker --version
# Docker version 24.0.6, build ed223bc
sudo usermod -a -G docker $(whoami)14.1.3.1 run docker compose
- /Users/bbest/My Drive/private/
msens_server_env-password.txt
sudo chown -R ubuntu:ubuntu /share
mkdir -p /share/github/MarineSensitivity
cd /share/github/MarineSensitivity
# clone server repo
git clone https://github.com/MarineSensitivity/server.git
cd server
# add password, used as $PASSWORD in docker-compose.yml
echo 'PASSWORD=******' > .env
# launch docker instances
sudo docker-compose up -d14.2 Docker compose
The Docker compose file is used to define and run multi-container Docker applications. Here is the docker-compose.yml file for the server pasted for convenience …
version: "3.9"
services:
caddy:
container_name: caddy
build: ./caddy
ports:
- 80:80
- 443:443
restart: unless-stopped
environment:
# Cloudflare Access team domain + application AUD tag for the signed-in
# preview.marinesensitivity.org vhost (caddy/Caddyfile jwtauth). Set both
# in the server .env once the Access application exists
# (server/cloudflare/). Unset, the vhost fails CLOSED: the defaults match
# no real JWT, so every request there gets 401 rather than the app.
CF_ACCESS_TEAM: ${CF_ACCESS_TEAM:-unconfigured}
CF_ACCESS_AUD: ${CF_ACCESS_AUD:-unconfigured}
# the restricted releases, as a regex alternation ("v8|v9"): the PUBLIC app
# vhost 302s /{ver}/scores|species|atlas for these to the review host
# instead of silently rendering the promoted release (or, for atlas, a
# bare 404 -- this host has no other route for /{ver}/atlas/ at all).
# Kept in the server .env beside PREVIEW_REVIEWERS_<VER>; DEPLOY_CADDY
# asserts it against versions.json. Unset, nothing matches and the old
# fallback behaviour stands.
PREVIEW_RESTRICTED_VERSIONS: ${PREVIEW_RESTRICTED_VERSIONS:-__none__}
volumes:
- ./caddy/Caddyfile:/etc/caddy/Caddyfile
# the preview host's routes, in their own file so caddy/test/run.sh can
# exercise the very same file behind a test jwtauth (single-file bind
# mount, same restart rule as the Caddyfile)
- ./caddy/preview_routes.caddy:/etc/caddy/preview_routes.caddy
# the version-path routes, shared by BOTH app vhosts (see the Caddyfile)
- ./caddy/app_version_routes.caddy:/etc/caddy/app_version_routes.caddy
# the atlas app's routes on the preview host, imported from
# preview_routes.caddy (single-file bind mount, same restart rule)
- ./caddy/atlas_preview_routes.caddy:/etc/caddy/atlas_preview_routes.caddy
- /share:/share
# the atlas-preview clone, explicitly read-only: caddy only ever serves
# from it, atlas-preview (below) is the only writer
- /share/atlas_preview:/share/atlas_preview:ro
- /share/caddy/data:/data
- /share/caddy/config:/config
rstudio:
container_name: rstudio
build:
context: ./rstudio
args:
# which msens the IMAGE carries. Default `main` = never stale; the deploy
# (rstudio/build.sh) passes the resolved commit instead, which both
# records what was built and busts Docker's layer cache. The running
# container additionally reconciles with the /share checkout at start
# (rstudio/cont-init/03_msens_from_share).
MSENS_REF: ${MSENS_REF:-main}
environment:
ROOT: 'true'
# The login is `rstudio`, and DEFAULT_USER must say so.
#
# rocker's /rocker_scripts/init_userconf.sh sets USER=${DEFAULT_USER}, but it
# only ever CREATES that account inside `if [ "$USERID" -ne 1000 ]` (or the
# rootless branch). At the default USERID=1000 neither runs, so the image's
# built-in `rstudio` user is all that exists — and line 167,
# echo "$USER:$PASSWORD" | chpasswd
# then targets an account that was never created.
#
# With DEFAULT_USER: admin that failed silently on every recreate: no `admin`
# to log in as, and `rstudio` still carrying whatever password the image was
# built with, so BOTH logins were rejected while .env looked perfectly
# correct. Verified by comparing the .env value against the shadow hash: NO
# MATCH.
#
# Do NOT "fix" this by setting USERID to force the rename — uid 1000 is what
# matches host `ubuntu`, and scripts/srv_render.sh renders as `-u 1000:1000`
# precisely because of it.
DEFAULT_USER: rstudio
PASSWORD: ${PASSWORD}
ADD: shiny
# usage-log Sheet endpoint (Apps Script /exec) for the Shiny apps'
# msens::ga_js() beacon. Set MSENS_LOG_URL in the server .env alongside
# PASSWORD; if unset the Sheet leg is a silent no-op and only GA4 receives
# events. The apps hold no credential — the endpoint is the whole secret,
# so it belongs in .env (untracked), never in the app source.
MSENS_LOG_URL: ${MSENS_LOG_URL:-}
# signs the version token a Shiny page embeds so the session renders the
# version its page was served for (msens::ver_token_*). Set in .env
# (`openssl rand -hex 32`); reaches the app processes via rocker's
# Renviron.site. Unset, msens uses a per-process random secret, which is
# still correct -- only a session straddling a process restart is asked
# to reload.
MS_TOKEN_SECRET: ${MS_TOKEN_SECRET:-}
# shared secret the PREVIEW app instance sends to the API as
# X-MS-Preview-Token, so /report will render a restricted (under-review)
# release for a signed-in reviewer and for nobody else. Same value in the
# plumber service below; unset = restricted reports refused everywhere.
MSENS_PREVIEW_TOKEN: ${MSENS_PREVIEW_TOKEN:-}
ports:
- 8787:8787 # rstudio
- 3838:3838 # shiny
restart: unless-stopped
# MEMORY CEILING (2026-09-24 outage): a Shiny Server app worker (uid 996 `shiny`) grew to
# 7.8 GB resident inside this container; with no cgroup limit and no swap the host spent an
# hour reclaiming page cache (ssh, apps, STAC, API all unreachable) before the OOM killer
# fired, and only an EC2 reboot ended it. With a ceiling, a runaway worker is killed INSIDE
# this container (its Shiny session errors; everything else keeps serving). 9 GB leaves
# ~6 GB for erddap/titiler/plumber/stac-api/caddy on the 16 GB host at their measured idle
# (~3.2 GB). memswap_limit == mem_limit: this container never spills into the host swap
# (host/swap.sh); swap exists so the REST of the host degrades instead of wedging.
mem_limit: 9g
memswap_limit: 9g
volumes:
- /share:/share
- /share/shiny_apps:/srv/shiny-server
# Shiny Server config with a SECOND server block (:3839) for the preview
# instance -- same app code, MS_PREVIEW=1 -- that only Caddy's signed-in
# preview.marinesensitivity.org vhost proxies to. Single-file bind mount:
# a change needs `docker compose restart rstudio` (inode rule, as Caddyfile).
- ./rstudio/shiny-server.conf:/etc/shiny-server/shiny-server.conf:ro
# the preview instance's site_dir: 3-line wrapper app.R per app that sets
# MS_PREVIEW=1 and shinyAppDir()s the real app; `touch restart.txt` here
# reloads it (DEPLOY_APPS does). Not published as a port on purpose.
- ./rstudio/shiny_apps_preview:/srv/shiny-server-preview
# Release notebooks rendered INSIDE this container publish to S3. The image
# carries the AWS CLI, but a CLI with no credentials just fails later and
# louder: `NoCredentials` at the publish step, after the build work is done.
# Read-only, and the container already runs as uid 1000 = host `ubuntu`,
# which owns these.
- /home/ubuntu/.aws:/home/rstudio/.aws:ro
# warm: keep the versions people actually open already built, so the first
# visitor after a deploy or a quiet night does not pay the ~13-17 s cold start
# (app_idle_timeout can only keep alive what someone already started). Reads
# the registry for WHICH versions — promoted + restricted — and hits the
# container ports directly, so no Cloudflare round trip and no Access token.
# See warm/warm.sh.
warm:
container_name: warm
image: alpine:3.22
restart: unless-stopped
depends_on: [rstudio]
entrypoint: ["/bin/sh", "-c"]
command: ["apk add --no-cache curl jq >/dev/null && exec /warm.sh"]
volumes:
- ./warm/warm.sh:/warm.sh:ro
# docs-preview: the rendered books of RESTRICTED docs releases. GitHub Pages
# cannot be gated, so the docs CI publishes restricted versions to the
# `gh-pages-preview` branch (public versions go to gh-pages as always) and
# Caddy's signed-in preview vhost serves this clone at /docs/{ver}/. The
# sidecar keeps the clone current by polling -- no inbound secret, no manual
# step; a docs push shows up within 5 min. Runs as uid 1000 (= host `ubuntu`)
# so nothing under /share ends up root-owned. The branch may not exist yet on
# a fresh install: the loop just retries.
docs-preview:
container_name: docs-preview
image: alpine/git:v2.49.1
restart: unless-stopped
user: "1000:1000"
environment:
HOME: /tmp
GIT_TERMINAL_PROMPT: "0"
entrypoint: ["/bin/sh", "-c"]
command:
- |
REPO=https://github.com/MarineSensitivity/docs.git
while true; do
if [ ! -d /repo/.git ]; then
git clone --quiet --branch gh-pages-preview --single-branch "$$REPO" /repo \
&& echo "cloned gh-pages-preview" \
|| echo "clone failed (gh-pages-preview may not exist yet); retrying in 5 min"
else
git -C /repo pull --ff-only --quiet || echo "pull failed; retrying in 5 min"
fi
sleep 300
done
volumes:
# HOST STATE: this directory must be owned by uid 1000 (`sudo chown
# 1000:1000 /share/docs_preview`). Docker auto-creates a missing bind-mount
# source as root:root, and the sidecar then cannot write its clone.
- /share/docs_preview:/repo
# atlas-preview: the STATIC BUILD of the atlas app, for RESTRICTED releases.
# GitHub Pages serves `gh-pages` publicly and cannot be gated, so the preview
# host's Caddy vhost (caddy/atlas_preview_routes.caddy) serves this clone at
# /{ver}/atlas/ instead -- the SAME dist/, the SAME commit, just a different
# door (atlas/CLAUDE.md: the build is base-relative and router-free, so it
# runs unmodified under either mount point). This sidecar keeps the clone
# current by polling the same `gh-pages` branch the public host reads -- no
# inbound secret, no manual step; an atlas push shows up within 5 min. Runs as
# uid 1000 (= host `ubuntu`) so nothing under /share ends up root-owned. The
# branch may not exist yet before atlas-0's first CI publish: the loop just
# retries.
atlas-preview:
container_name: atlas-preview
image: alpine/git:v2.49.1
restart: unless-stopped
user: "1000:1000"
environment:
HOME: /tmp
GIT_TERMINAL_PROMPT: "0"
entrypoint: ["/bin/sh", "-c"]
command:
- |
REPO=https://github.com/MarineSensitivity/atlas.git
while true; do
if [ ! -d /repo/.git ]; then
git clone --quiet --branch gh-pages --single-branch "$$REPO" /repo \
&& echo "cloned gh-pages" \
|| echo "clone failed (gh-pages may not exist yet); retrying in 5 min"
else
git -C /repo pull --ff-only --quiet || echo "pull failed; retrying in 5 min"
fi
sleep 300
done
volumes:
# HOST STATE: this directory must be owned by uid 1000 (`sudo chown
# 1000:1000 /share/atlas_preview`). Docker auto-creates a missing
# bind-mount source as root:root, and the sidecar then cannot write its
# clone.
- /share/atlas_preview:/repo
# plumber: DuckDB-backed custom API (postgres stack retired 2026-06). Serves
# the OBIS H3 summary endpoint /h3 (obisindicators::obis_h3t_sql) from the same
# store as the h3t tile service. Caddy already routes
# api.marinesensitivity.org -> plumber:8888. No postgis dependency: the
# postgres-backed endpoints degrade (see api/plumber.R lazy `con`).
plumber:
container_name: plumber
build: ./plumber
environment:
MSENS_OBIS_DUCKDB: /share/data/obis/obis_h3.duckdb
# the other half of the review gate's report check: /report renders a
# restricted (under-review) release only for a caller presenting this
# secret in X-MS-Preview-Token -- i.e. the preview Shiny instance. Must
# match the rstudio service's value; unset = restricted reports refused.
MSENS_PREVIEW_TOKEN: ${MSENS_PREVIEW_TOKEN:-}
ports:
- 8888:8888 # api
volumes:
- /share:/share
restart: unless-stopped
# --- postgres stack retired (2026-06) to free resources for OBIS h3 build ---
# plumber:
# container_name: plumber
# build: ./plumber
# ports:
# - 8888:8888 # api
# restart: unless-stopped
# volumes:
# - /share:/share
# depends_on:
# - postgis
# postgis:
# container_name: postgis
# image: postgis/postgis:latest
# environment:
# POSTGRES_DB: msens
# POSTGRES_USER: admin
# POSTGRES_PASSWORD: ${PASSWORD}
# ANON_PASSWORD: ${ANON_PASSWORD}
# PGDATA: /share/postgis/data
# volumes:
# - ./postgis/init.sh:/docker-entrypoint-initdb.d/init.sh
# - /share:/share
# - /share/postgis:/var/lib/postgresql
# restart: unless-stopped
# healthcheck:
# test: 'exit 0'
# ports:
# - 5432:5432
# pgadmin:
# container_name: pgadmin
# image: dpage/pgadmin4:8.14
# restart: always
# environment:
# PGADMIN_DEFAULT_EMAIL: ben@ecoquants.com
# PGADMIN_DEFAULT_PASSWORD: ${PASSWORD}
# PGADMIN_LISTEN_PORT: 8088
# ports:
# - 8088:8088
# volumes:
# - /share/pgadmin:/var/lib/pgadmin
# depends_on:
# - postgis
# pgbkups:
# container_name: pgbkups
# image: prodrigestivill/postgres-backup-local
# restart: always
# user: postgres:postgres
# volumes:
# - /share/postgis_backups:/backups
# links:
# - postgis
# depends_on:
# - postgis
# environment:
# - POSTGRES_HOST=postgis
# - POSTGRES_DB=msens
# - POSTGRES_USER=admin
# - POSTGRES_PASSWORD=${PASSWORD}
# - POSTGRES_EXTRA_OPTS=-Z6 --blobs
# - SCHEDULE=@daily
# - HEALTHCHECK_PORT=8088
# - BACKUP_KEEP_DAYS=2
# - BACKUP_KEEP_WEEKS=1
# - BACKUP_KEEP_MONTHS=2
# tile:
# container_name: tile
# environment:
# DATABASE_URL: 'postgresql://admin:${PASSWORD}@postgis:5432/msens'
# image: pramsey/pg_tileserv:latest
# depends_on:
# - postgis
# ports:
# - 7800:7800
# tilecache:
# container_name: tilecache
# image: varnish:latest
# volumes:
# - /share:/share
# ports:
# - 6081:6081
# environment:
# VARNISH_BACKEND_HOST: tile
# VARNISH_BACKEND_PORT: 7800
# VARNISH_HTTP_PORT: 6081
# restart: always
# depends_on:
# - "tile"
# --- end postgres stack ---
# rest:
# container_name: rest
# environment:
# PGRST_DB_URI: 'postgresql://anon:${ANON_PASSWORD}@postgis:5432/msens'
# PGRST_OPENAPI_SERVER_PROXY_URI: http://127.0.0.1:3000
# PGRST_DB_ANON_ROLE: anon
# image: postgrest/postgrest
# depends_on:
# - postgis
# ports:
# - "3000:3000"
# swagger:
# container_name: swagger
# image: swaggerapi/swagger-ui
# depends_on:
# - rest
# ports:
# - "8080:8080"
# expose:
# - "8080"
# environment:
# API_URL: https://rest.MarineSensitivity.org/
titiler:
container_name: titiler
build: ./titiler
environment:
PORT: 8000
# WORKERS_PER_CORE: 1
MSENS_CELLID_COG: /share/data/derived/r_cellid.tif
MSENS_DUCKDB: /share/data/big/latest/sdm.duckdb
MSENS_MAX_ROWS: "1000000"
MSENS_LRU_SIZE: "128"
volumes:
- /share:/share
ports:
- "8000:8000"
restart: always
# v8 parallel serving: a tiny view-only DuckDB whose model_cell is a VIEW over the
# partitioned Parquet on S3 (marine-atlas/v8), rendered on the global [-180,180] cell COG.
# v7 titiler above stays fully live (A/B). serve.duckdb is KB — no multi-GB rsync.
# Stock titiler: /cog serves the precomputed COGs every app layer now uses.
#
# The custom /msens cells factory (DuckDB query -> tile) is RETIRED but KEPT:
# set MSENS_FACTORY=1 to mount it for exploratory on-the-fly queries. It is off
# by default because nothing reaches it — the scores app's SQL fallback cannot
# fire (every metric on v1-v8 has a COG) and the species app's needed only the
# 192 taxa that now have merged COGs. Verified equivalent first: 100/105 tiles
# byte-identical, the rest differing by 3-6 px of 262,144 from float32 storage.
#
# Deliberately NOT proxied through titilecache (Varnish): a query-driven tile
# is not cache-friendly the way an immutable COG is, and caching an exploratory
# endpoint mostly serves stale answers to whoever is iterating on the query.
# The MSENS_* vars below stay so enabling the factory needs no other change.
titiler-v8:
container_name: titiler-v8
build: ./titiler
environment:
PORT: 8000
MSENS_CELLID_COG: /share/data/derived/r_cellid_global.tif
MSENS_DUCKDB: /share/data/big/v8/serve.duckdb
MSENS_MAX_ROWS: "1000000"
MSENS_LRU_SIZE: "128"
volumes:
- /share:/share
ports:
- "8001:8000"
restart: always
# stac-api: a SEARCHABLE STAC API with one Item per model, complementing (not
# replacing) the static catalog at file.marinesensitivity.org/stac.
#
# The static catalog is dataset-level: one Item per collection whose assets are
# S3 PREFIXES (…/native/am_native/, which 403s) and whose bbox is the whole
# dataset envelope — so it can answer neither "the asset for this model" nor
# "which models cover this area". This serves 25,643 per-model Items with real
# asset hrefs and real footprints.
#
# Backend reads stac-geoparquet via DuckDB — the same Parquet-on-DuckDB shape as
# the rest of this stack. Items and collection.json come from the tree that
# workflows/publish_stac_api.qmd builds and rsyncs; PARQUET_URLS_JSON (collection
# id -> parquet path) is passed by that notebook from parquet_urls.json, so
# adding a dataset needs no edit here.
#
# Pinned to an exact upstream commit built straight from GitHub — the project is
# marked EXPERIMENTAL by both its repo and the stac-fastapi docs (pgstac and
# Elasticsearch are the production-ready backends), so a moving default branch
# is not something to serve from. Treat this as a trial surface, not load-bearing.
stac-api:
container_name: stac-api
build: ./stac-api # upstream pinned + ST_Intersects(WKB) patch; see its Dockerfile
environment:
APP_HOST: 0.0.0.0
APP_PORT: "8084"
ENVIRONMENT: production
WEB_CONCURRENCY: "4"
BACKEND: duckdb
STAC_FILE_PATH: /app/stac_collections
PARQUET_URLS_JSON: ${PARQUET_URLS_JSON:?written to .env by workflows/publish_stac_api.qmd}
volumes:
# read-only: the API must never mutate the published catalog. The version is STAC_API_VER in .env,
# written by publish_stac_api.qmd beside PARQUET_URLS_JSON: a hardcoded v8 here once served v9's
# PARQUET_URLS_JSON against v8's tree, and every collection answered "No Parquet configured" (2026-10-07)
- /share/data/derived/stac-api/${STAC_API_VER:?written to .env by workflows/publish_stac_api.qmd}:/app/stac_collections:ro
ports:
- "8085:8084"
restart: unless-stopped
command: bash -c "python -m stac_fastapi.duckdb.app"
titilecache:
container_name: titilecache
image: varnish:latest # 7.4.2 # last updated: 2023-12-26
volumes:
- /share:/share
- "./varnish/titiler.vcl:/etc/varnish/default.vcl:ro"
ports:
- 6082:6082
environment:
VARNISH_BACKEND_HOST: titiler
VARNISH_BACKEND_PORT: 8000
VARNISH_HTTP_PORT: 6082
command: "-p default_keep=604800" # 7d, matches vcl TTL
restart: always
depends_on:
- "titiler"
# h3t: OBIS biodiversity-by-hex tile factory (DuckDB SELECT -> h3j JSON tiles)
h3t:
container_name: h3t
build: ./h3t
environment:
H3T_DBS: "obis:/share/data/obis/obis_h3.duckdb"
H3T_DEFAULT_DB: obis
H3T_MAX_ROWS: "50000"
# common filtered maps now hit precomputed idx_h3 / idx_h3_taxon layers;
# these caps are the stopgap for the remaining live occ_h3 paths (year
# filters, multi-value / fine-rank taxa). raise the timeout above the
# app-side client timeout in apps/h3-db/app.R.
H3T_STMT_TIMEOUT_MS: "8000"
H3T_THREADS: "4" # cap serving CPU (don't peg all cores)
H3T_MEMORY_LIMIT: "6GB" # cap serving RAM (occ_h3 rollups can spill)
# /h3t/subtree + /h3t/taxon (obis-hex app): own DuckDB instance, capped
# separately; see h3t/README.md "subtree endpoint"
H3T_SUBTREE_THREADS: "2"
H3T_SUBTREE_MEMORY_LIMIT: "1GB"
H3T_SUBTREE_TIMEOUT_S: "60"
H3T_SUBTREE_MAX_CELLS: "200000"
volumes:
- /share:/share
ports:
- "8889:8889"
restart: always
h3tcache:
container_name: h3tcache
image: varnish:latest # 7.4.2 # last updated: 2023-12-26
volumes:
- /share:/share
- "./varnish/h3t.vcl:/etc/varnish/default.vcl:ro"
ports:
- 6083:6083
environment:
VARNISH_BACKEND_HOST: h3t
VARNISH_BACKEND_PORT: 8889
VARNISH_HTTP_PORT: 6083
command: "-p default_keep=604800" # 7d, matches vcl TTL
restart: always
depends_on:
- "h3t"14.3 DNS
The domain name server (DNS) records are managed by SquareSpace. The subdomains point to the server on Amazon at 100.25.173.0, whereas the main website is hosted by Github servers, per Managing a custom domain for your GitHub Pages site - GitHub Docs.
| Host | Type | Data |
|---|---|---|
| @ | A | 185.199.111.153 |
| @ | A | 185.199.110.153 |
| @ | A | 185.199.109.153 |
| @ | A | 185.199.108.153 |
| api | A | 100.25.173.0 |
| app | A | 100.25.173.0 |
| file | A | 100.25.173.0 |
| msens1 | A | 100.25.173.0 |
| pgadmin | A | 100.25.173.0 |
| pmtiles | A | 100.25.173.0 |
| rstudio | A | 100.25.173.0 |
| shiny | A | 100.25.173.0 |
| tile | A | 100.25.173.0 |
| tilecache | A | 100.25.173.0 |
| titiler | A | 100.25.173.0 |
| titilecache | A | 100.25.173.0 |
| www | CNAME | marinesensitivity.org |
14.4 Caddyfile
The Caddyfile parameterizes the reverse proxying between the external subdomains and the Docker’s internal ports. Here is the Caddyfile pasted for convenience …
# docker exec caddy caddy reload --config /etc/caddy/Caddyfile
{
order pmtiles_proxy before file_server
# jwtauth (ggicci/caddy-jwt, built into caddy/Dockerfile) must run before any
# handler on the vhosts that use it -- see preview.marinesensitivity.org
order jwtauth before basic_auth
}
(cors) {
@origin header Origin *
header @origin {
Access-Control-Allow-Origin "*"
Access-Control-Request-Method GET
}
}
# CORS for the storage.* redirect to S3 (2026-10-08): the 302 itself must carry
# the headers, and OPTIONS must be answered here, because a browser preflights
# before it ever follows the redirect. Not (cors): that one only answers when
# the request has an Origin and adds no Expose/Methods/Max-Age.
(s3_cors) {
header {
Access-Control-Allow-Origin "*"
Access-Control-Allow-Methods "GET, HEAD"
Access-Control-Expose-Headers "Content-Range, Content-Length, Accept-Ranges, ETag"
Access-Control-Max-Age "3600"
}
}
# Every vhost logs (2026-10-02 egress review: only storage/preview/file did, so
# nobody could say who was crawling the other ~17). Every byte served is EC2
# egress, so the log is what turns a surprise bill into a name.
# import access_log <name> -> /share/logs/caddy/<name>.log (JSON, rotated)
(access_log) {
log {
output file /share/logs/caddy/{args[0]}.log {
roll_size 64MiB
roll_keep 14
roll_keep_for 2160h
}
format json
}
}
# Crawler policy for the DATA hosts (tiles, COGs, API, files). These are used by
# browsers through the apps, never meant to be walked: tell well-behaved crawlers
# to stay out, and refuse the AI/SEO scrapers outright.
(robots_none) {
handle /robots.txt {
header Content-Type "text/plain; charset=utf-8"
respond `User-agent: *
Disallow: /
` 200
}
}
# 403 for AI-training / SEO scrapers by User-Agent (case-insensitive substring).
# Deliberately NOT listed: Claude-User, ChatGPT-User -- a person asking an
# assistant to fetch one URL is wanted. Arguments are the paths exempt from the
# block (always /robots.txt; the file host also exempts its small browsable
# trees). This is a `handle`, not a bare `respond`, so that it also runs on a
# vhost made of sibling `handle` blocks (file), where a respond would sort AFTER
# them and never fire.
(no_ai_bots) {
@ai_bots {
header_regexp ai_bot User-Agent (?i)(GPTBot|OAI-SearchBot|ClaudeBot|Claude-SearchBot|anthropic-ai|Amazonbot|Bytespider|CCBot|PerplexityBot|Google-Extended|meta-externalagent|AhrefsBot|SemrushBot|MJ12bot|DotBot)
not path {args[:]}
}
handle @ai_bots {
header Content-Type "text/plain; charset=utf-8"
respond "Automated crawling of these data hosts is not permitted. See https://marinesensitivity.org/ for the catalog." 403
}
}
api.marinesensitivity.org {
import access_log api
import robots_none
import no_ai_bots /robots.txt
reverse_proxy plumber:8888
}
file.marinesensitivity.org {
import cors
# This host had no access log and no robots.txt while it listed ~80 GB of
# data as browsable directories (2026-10-02 egress review). Every byte it
# serves is VM egress, so: log it, tell crawlers to stay out of the data,
# and serve files by exact URL only -- no directory listings on the data
# trees. The browsable front door for data is the S3 index at
# storage.marinesensitivity.org; /stac/ and /branding/ stay browsable (small).
import access_log file
import no_ai_bots /robots.txt /stac/* /branding/*
handle /robots.txt {
header Content-Type "text/plain; charset=utf-8"
respond `# The STAC catalog and the brand set: crawl away. Data files: please do not.
User-agent: *
Crawl-delay: 10
Allow: /stac/
Allow: /branding/
Disallow: /
` 200
}
# the brand set, served straight from the repo checkout: a `git pull` on the
# server publishes it, so there is no copy step to forget. Used by the
# Cloudflare Access login page (which takes a logo URL), the docs, the apps
# and any deck. See server/branding/README.md.
handle_path /branding/* {
root * /share/github/MarineSensitivity/server/branding
header Cache-Control "public, max-age=3600"
file_server browse
}
# raw pmtiles for client-side PMTiles protocol (add_pmtiles_source)
handle_path /pmtiles/* {
root * /share/data/derived/pmtiles
file_server
}
# The per-model PMTiles of v8 and v9 were served from /share/data/pmtiles_native
# (moved out of derived/ on 2026-10-02 after the BOEM mirror's rclone loop re-read
# them hourly, ~150 GB/day) until the asset-store prune of 2026-10-07 removed them:
# every published v8/v9 `native_asset` row names a store object on S3
# (native/{ds_key}/{hash}.pmtiles). The old URLs answer 410 here rather than
# falling through to /share/data/derived/pmtiles, so a stray copy in the old
# place can never be served again (caddy/test/file_host_local.sh proves it).
handle /pmtiles/v8/* {
respond "Gone: per-model PMTiles are served from the asset store; see the release's native_asset table" 410
}
handle /pmtiles/v9/* {
respond "Gone: per-model PMTiles are served from the asset store; see the release's native_asset table" 410
}
# legacy versioned pmtiles routes (backwards compat for old apps)
handle_path /pmtiles/v4/* {
root * /share/data/derived/v4/pmtiles
file_server
}
handle_path /pmtiles/v3/* {
root * /share/data/derived/v3/pmtiles
file_server
}
# stac catalog (sdm extension) — static JSON tree
handle_path /stac/* {
root * /share/data/derived/stac
file_server browse
}
# versioned derived data (geoparquet, cogs, gpkg) referenced by stac assets
handle_path /derived/* {
root * /share/data/derived
file_server
}
# force reports to download (especially HTML which otherwise renders inline)
handle_path /reports/* {
root * /share/public/reports
header Content-Disposition "attachment"
file_server
}
handle {
root * /share/public
file_server {
precompressed zstd br gzip
}
}
}
# Browsable front door for the PUBLIC S3 bucket -- folder PAGES only.
#
# Why this exists: S3 serves OBJECTS, not directories. A URL ending in "/" 404s
# unless an object literally has that key, and anonymous ListBucket is DENIED on
# this bucket (403 AccessDenied), so a browser cannot enumerate anything. The
# workflows notebook publish_storage_index.qmd generates a real index.html per
# directory; this vhost rewrites folder URLs to them:
#
# storage.marinesensitivity.org/marine-atlas/v8/ -> .../v8/index.html (proxied)
# storage.marinesensitivity.org/marine-atlas/latest.txt -> 302 to the object
#
# ONLY the small pages transit this server: folder URLs, */index.html and
# */README.md. EVERY OTHER object is a 302 to the bucket's own HTTPS endpoint,
# so none of its bytes -- Range requests included, a redirect ignores Range --
# is billed as VM egress (it used to be proxied: 18 GB across ~90,000 objects,
# 2026-10-02 egress review). The index pages already link files at that endpoint
# directly. 302, not 301, because the store may move to another provider and a
# cached 301 would pin clients to the old one.
#
# The redirect carries CORS (2026-10-08): the apps' pmtiles / COG / Parquet range
# requests to this host failed with status 0 because the bare 302 (and a bare 302
# on the OPTIONS preflight) had no Access-Control-* headers. The 302 now has
# them (snippet s3_cors) and OPTIONS is a 204 here; the bucket itself answers
# with Access-Control-Allow-Origin: * once the browser lands on it.
#
# Named "storage" rather than "s3" so the URL survives a move to another
# provider.
# storage.oceanmetrics.io is the same bucket under the Ocean Metrics brand (the
# gazetteer/ prefix is published there first; marine-atlas/ keeps its old links).
storage.marinesensitivity.org, storage.oceanmetrics.io {
encode zstd gzip
import access_log storage
# robots.txt inline, BEFORE the bucket handler, so it cannot be lost by an
# index regeneration. The catalog pages are welcome in search results; the
# objects behind them are not (they are now redirected to S3, but a crawler
# that follows the redirect still walks 90,000 of them).
handle /robots.txt {
header Content-Type "text/plain; charset=utf-8"
respond `# Catalog pages: crawl away. Data objects: please do not.
User-agent: *
Crawl-delay: 10
Disallow: /*.tif$
Disallow: /*.parquet$
Disallow: /*.duckdb$
Disallow: /*.pmtiles$
Disallow: /*.gz$
Disallow: /*.zip$
` 200
}
# ONLY the published atlas. The same bucket holds backups/ -- including a
# multi-GB database dump -- which is technically public but must not be
# advertised or reachable here, even by exact path. This allow-list is the
# server-side half of the restriction the index generator applies.
@atlas path_regexp atlas ^/(marine-atlas|gazetteer)(/|$)
handle @atlas {
# CORS preflight, answered here (a redirect on OPTIONS fails the preflight).
# Suffix ranges (bytes=-N, how Parquet readers fetch the footer) and
# If-Match / If-None-Match are not CORS-safelisted, so they DO preflight.
@preflight method OPTIONS
handle @preflight {
import s3_cors
header Access-Control-Allow-Headers "Range, If-Match, If-None-Match, If-Range, Content-Type"
respond 204
}
# folder URL -> the index.html object standing in for a listing
@dir path */
handle @dir {
rewrite * {path}index.html
reverse_proxy https://s3.us-east-1.amazonaws.com {
header_up Host s3.us-east-1.amazonaws.com
rewrite /oceanmetrics.io-public{uri}
}
}
# the pages themselves: small, and what a browser needs on this host
@page path */index.html */README.md
handle @page {
reverse_proxy https://s3.us-east-1.amazonaws.com {
header_up Host s3.us-east-1.amazonaws.com
rewrite /oceanmetrics.io-public{uri}
}
}
# everything else is a data object: hand the client to S3, move no bytes
handle {
import s3_cors
header Cache-Control "public, max-age=86400"
redir https://s3.us-east-1.amazonaws.com/oceanmetrics.io-public{uri} 302
}
}
# the bare host lands on the atlas index
redir / /marine-atlas/ 302
# anything else: refuse, and say where to go
handle {
header Content-Type "text/plain; charset=utf-8"
respond `Not found.
Browsable: /marine-atlas/ and /gazetteer/
Start here: https://storage.marinesensitivity.org/marine-atlas/
` 404
}
}
pgadmin.marinesensitivity.org {
import access_log pgadmin
reverse_proxy pgadmin:8088
}
# rest.marinesensitivity.org {
# reverse_proxy rest:3000
# }
# searchable STAC API (one Item per model) — complements the STATIC catalog served
# from file.marinesensitivity.org/stac, which stays exactly as it is.
stac-api.marinesensitivity.org {
import access_log stac-api
import robots_none
reverse_proxy stac-api:8084
}
rstudio.marinesensitivity.org {
import access_log rstudio
reverse_proxy rstudio:8787
}
shiny.marinesensitivity.org {
import access_log shiny
reverse_proxy rstudio:3838
}
app.marinesensitivity.org {
import access_log app
# ---- MST app cutover -------------------------------------------------
#
# /scores and /species are now ONE app that renders any published release
# from ?ver=. Every former per-version instance redirects to it rather than
# 404ing, so old links, bookmarks and anything in a published report keep
# resolving — to the SAME release they always showed.
#
# The v7 apps were `mapgl` (scores) and `mapsp` (species); v1-v6 were
# `mapgl_v{n}` / `mapsp_v{n}`; v6 and v8 also had `scores_v{n}` /
# `species_v{n}` aliases. All of them land on /scores or /species with the
# version they represent.
#
# 301 (permanent): these instances are not coming back, and a permanent
# redirect lets browsers and crawlers stop asking.
#
# `&{query}` IS THE POINT OF THE REDIRECT. A redir target that carries its own
# query string REPLACES the incoming one, so these rules used to answer
# /mapsp/?mdl_seq=1434 with /species/?ver=v7 -- dropping the very thing the
# link was about. The app then opened on its default taxon, which is why every
# deep link in the final report, and the ones in issue #5, silently showed the
# leatherback turtle. Every published per-species link is of this form, so a
# redirect that keeps the path and loses the id is worse than a 404: it looks
# like it worked (MarineSensitivity/apps#6).
#
# An empty {query} leaves a harmless trailing "&", which parses to nothing.
@scores_ver path_regexp scv ^/(?:mapgl|scores)_(v[0-9]+[a-z]?)(?:/(.*))?$
redir @scores_ver /{re.scv.1}/scores/?{query} 301
@species_ver path_regexp spv ^/(?:mapsp|species)_(v[0-9]+[a-z]?)(?:/(.*))?$
redir @species_ver /{re.spv.1}/species/?{query} 301
# the unsuffixed v7 instances: `mapgl`/`mapsp` were what /scores and
# /species pointed at, so they carry v7 explicitly
@scores_v7 path_regexp sc7 ^/mapgl(?:/(.*))?$
redir @scores_v7 /v7/scores/?{query} 301
@species_v7 path_regexp sp7 ^/mapsp(?:/(.*))?$
redir @species_v7 /v7/species/?{query} 301
# A RESTRICTED release asked for on the PUBLIC host goes to the review host
# (2026-08-28). The app process here cannot resolve a restricted version
# (msens::atlas_allow_access), so /v8/scores/ used to render the promoted
# release with no hint -- fail-closed, but from a published link it simply
# looked like the wrong version (the homepage's v8 links "oddly ended up on
# v7"). Every spelling funnels through the path form first -- the retirement
# 301s above, the ?ver= 301s in app_version_routes.caddy -- so this ONE
# matcher catches them all and {uri} carries the query (deep links keep
# their mdl_key). 302, not 301: a version stops being restricted when it is
# released, and a cached 301 would keep sending visitors to the review host.
#
# `atlas` joined this matcher 2026-09-24 (atlas-9): the static atlas app has
# no canonical URL on THIS host (its public home is marinesensitivity.org/
# atlas/?ver=, GitHub Pages -- a different origin this Caddyfile does not
# serve at all), so nothing here would otherwise answer /{ver}/atlas/ except
# falling through to `reverse_proxy rstudio:3838`, which does not know that
# path. Covering it keeps the same guarantee scores/species already have --
# a reviewer who guesses the /v9/atlas/ shape by analogy lands on the review
# host, never a 404 that looks like "this version doesn't have an atlas" --
# and it is what sends a restricted version's atlas request to the ONE place
# that can actually serve it (preview_routes.caddy's atlas import).
#
# PREVIEW_RESTRICTED_VERSIONS is a regex alternation of the restricted
# versions ("v8|v9") from the server .env -- the same manual step as
# PREVIEW_REVIEWERS_<VER> and CF_ACCESS_AUD -- and DEPLOY_CADDY asserts it
# against versions.json before restarting. Unset, it matches nothing and the
# prior behaviour (fall back to the promoted release) stands.
@restricted_app path_regexp rapp ^/(?:{$PREVIEW_RESTRICTED_VERSIONS:__none__})/(?:scores|species|atlas)(?:/.*)?$
redir @restricted_app https://preview.marinesensitivity.org{uri} 302
# THE VERSION IS THE PATH here too (2026-08-27). One definition, shared with
# the preview vhost, so the two hosts cannot drift: /v7/scores/ is canonical,
# /scores/?ver=v7 301s to it, and the app reads X-MS-Version. A bare /scores/
# (no version) still resolves the promoted release inside the app -- Caddy
# cannot read latest.txt, and the app canonicalises the URL once it knows.
import app_version_routes.caddy rstudio:3838
# X-MS-User (the signed-in reviewer's identity) and X-MS-Version (the version
# from the preview host's URL path) are set by the PREVIEW vhost from things
# it has verified. Both are meaningful only there; a client must not be able
# to supply either here, where the version comes from ?ver= and the policy
# refuses restricted releases anyway.
request_header -X-MS-User
request_header -X-MS-Version
reverse_proxy rstudio:3838
}
# The signed-in PREVIEW host: restricted (under-review) releases of the apps and
# docs, for SDM data providers and BOEM/NOAA reviewers.
#
# How the gate works (workflows/.claude/plans/2026-08-15 pre-release review gate …):
# 1. DNS for this ONE hostname is proxied through Cloudflare, and a Cloudflare
# Access application in front of it lets a request through only after the
# visitor signs in (emailed one-time PIN, Google). Access then stamps every
# request -- page GETs, static assets, the Shiny websocket upgrade -- with
# Cf-Access-Jwt-Assertion, an RS256 JWT for this app's AUD.
# 2. jwtauth below is the ORIGIN-SIDE check Cloudflare documents: verify that
# JWT against the team's JWKS + issuer + audience. A request that reaches
# this box without going through Access (origin IP, wrong hostname, a
# forged header) has no valid JWT and gets 401 -- so proxying only this
# hostname is safe.
# 3. What is behind it (caddy/preview_routes.caddy): the PREVIEW Shiny Server
# instance (rstudio:3839, the same app code as /scores + /species with
# MS_PREVIEW=1 -- see rstudio/shiny-server.conf) at /{ver}/scores/ and
# /{ver}/species/ -- THE VERSION IS THE PATH, so Access can hold one
# reviewer policy per version -- the rendered books of restricted docs
# versions at /docs/{ver}/ (docs CI publishes those to the gh-pages-preview
# branch, cloned at /share/docs_preview by the docs-preview sidecar), and a
# landing page.
#
# CF_ACCESS_TEAM / CF_ACCESS_AUD come from the server .env. Their DEFAULT is a
# value nothing can match, so an unconfigured deploy fails CLOSED (401 for
# everyone) rather than open. Which versions are restricted is NOT known here:
# that is `access` in versions.json, enforced by the app process's MS_PREVIEW.
preview.marinesensitivity.org {
import access_log preview
jwtauth {
jwk_url https://{$CF_ACCESS_TEAM:unconfigured}.cloudflareaccess.com/cdn-cgi/access/certs
from_header Cf-Access-Jwt-Assertion
from_cookies CF_Authorization
issuer_whitelist https://{$CF_ACCESS_TEAM:unconfigured}.cloudflareaccess.com
audience_whitelist {$CF_ACCESS_AUD:unconfigured}
# A HUMAN's Access JWT carries `email`; a SERVICE TOKEN's carries
# `common_name` (its client id) and no email at all. caddy-jwt needs to
# find one of these or it rejects the request -- which is how the release
# notebook's own check token got a 401 from the ORIGIN after Cloudflare
# had already admitted it. (Cloudflare rewrites `server:` on proxied
# responses, so `server: cloudflare` on that 401 was not evidence that
# Cloudflare produced it; the body was this vhost's own error page.)
user_claims email common_name sub
}
# the routes themselves live in a file the functional test can import too
# (caddy/test/run.sh) -- see it for the URL scheme: the version is the PATH
import /etc/caddy/preview_routes.caddy
# jwtauth answers a bare 401 with an EMPTY body -- a blank page in a browser,
# indistinguishable from "broken". Say what this host is instead. (Cloudflare
# Access intercepts before the origin, so once it is in front a human never
# sees this; it is what a request that BYPASSED Access gets.)
handle_errors {
@unauth expression {http.error.status_code} == 401
handle @unauth {
header Content-Type "text/html; charset=utf-8"
respond <<HTML
<!doctype html><meta charset="utf-8"><meta name="robots" content="noindex">
<title>Marine Sensitivity — preview (sign-in required)</title>
<style>body{font:16px/1.5 system-ui,sans-serif;max-width:40rem;margin:4rem auto;padding:0 1.25rem}</style>
<h1>Marine Sensitivity — preview</h1>
<p>This host serves <b>restricted, pre-release</b> versions of the apps and documentation
to invited reviewers, and requires sign-in through the project's access gateway —
which is not yet in front of this address, so nothing here can be viewed at the moment.</p>
<p>Public releases: <a href="https://app.marinesensitivity.org/scores/">Scores</a> ·
<a href="https://app.marinesensitivity.org/species/">Species</a> ·
<a href="https://marinesensitivity.org/docs/">Documentation</a>.</p>
HTML 401
}
}
}
shiny.oceanmetrics.io {
import access_log shiny-oceanmetrics
reverse_proxy rstudio:3838
}
# swagger.marinesensitivity.org {
# reverse_proxy swagger:8080
# }
tile.marinesensitivity.org {
import access_log tile
import robots_none
import no_ai_bots /robots.txt
reverse_proxy tile:7800
}
tilecache.marinesensitivity.org {
import access_log tilecache
import robots_none
import no_ai_bots /robots.txt
reverse_proxy tilecache:6081
}
titiler.marinesensitivity.org {
import access_log titiler
import robots_none
import no_ai_bots /robots.txt
reverse_proxy titiler:8000
}
titiler-v8.marinesensitivity.org {
import access_log titiler-v8
import robots_none
import no_ai_bots /robots.txt
reverse_proxy titiler-v8:8000
}
titilecache.marinesensitivity.org {
import access_log titilecache
import robots_none
import no_ai_bots /robots.txt
reverse_proxy titilecache:6082
}
h3t.marinesensitivity.org {
import access_log h3t
import robots_none
import no_ai_bots /robots.txt
reverse_proxy h3t:8889
}
h3tcache.marinesensitivity.org {
import access_log h3tcache
import robots_none
import no_ai_bots /robots.txt
reverse_proxy h3tcache:6083
}
pmtiles.marinesensitivity.org {
import access_log pmtiles
import cors
import robots_none
import no_ai_bots /robots.txt
# server-side tile decoding: ZXY + TileJSON (for QGIS, leaflet, etc.)
handle_path /tiles/* {
pmtiles_proxy {
bucket file:///share/data/derived/v4/pmtiles
# note: v3 pmtiles also available at /share/data/derived/v3/pmtiles
cache_size 256
public_url https://pmtiles.marinesensitivity.org/tiles
}
}
}
# erddap.oceanmetrics.io: the vhost lives in its own repo (oceanmetrics/erddap,
# checked out under /share/github, which this container mounts); the erddap
# compose joins this stack's `server_default` network so `erddap:8080` resolves.
import /share/github/oceanmetrics/erddap/caddy/Caddyfile.erddap
14.5 Services
The server is running the following services. Subdomains with 🔒 are internal (admin / development); others are public.
14.5.1 App + data plane
app / shiny
interactive Shiny applications — proxied to therstudiocontainer’s Shiny Server on internal port 3838. There are now two applications,/scoresand/species, each rendering any published release from the URL path (e.g./v8/scores/). The former per-version instances (mapgl,mapsp,mapgl_v1…mapsp_v6,scores_v6/v8,species_v6/v8) are retired, with Caddy issuing a 301 to the equivalent path so every existing link — including any printed in a published report or an older?ver=deep link — still resolves to the release it always showed.
Shiny Server docsfile
static file server for PMTiles archives, report downloads, reference GeoPackages — served by Caddy’sfile_serverdirective directly from the/sharebind-mount. This replaces the oldpg_tileservvector-tile path for Program Areas, Ecoregions, Planning Areas, protractions, blocks and aliquots; each layer is a single.pmtilesfile that the browser range-reads for only the tiles it needs. It no longer hosts the per-species range tiles (thepmtiles/{v8,v9}/copies): from the asset-store migration (2026-10) those are served from the S3 store (native/{ds_key}/{hash}.pmtiles, Section 15.3), which answers range requests with CORS. The file host keeps serving the zone and reference layers, report downloads and reference GeoPackages, and the species copies are retained there until the prune that follows the migration.titiler + titiler-v8 🟢
TiTiler serving the published raster assets.
Every map layer the applications draw is now a pre-rendered Cloud-Optimized GeoTIFF read through titiler’s stock/cogroutes — score surfaces per metric × subregion, and per-model distributions.titiler-v8runs in parallel with the v7 instance, reading a kilobyte-sized view-only DuckDB whosemodel_cellis a view over the partitioned Parquet on S3, so no multi-gigabyte database is copied to the server.titilecache (Varnish) 🔒
Fronts the v7titileronly. The v8 service is deliberately not proxied through it: an immutable COG is already cacheable by URL, and the one endpoint that would benefit is retired (below).api
custom RplumberAPI for programmatic access to DuckDB — see Chapter 17
14.5.2 Admin
rstudio 🔒
integrated development environment (IDE) to code and debug directly on the server.
Posit RStudio Serverpgadmin 🔒 — PostgreSQL administration interface, commented out along with the PostgreSQL services it managed (see Retired, below).
14.5.3 Retired
The custom
/msenstile factory — kept in the image but off by default, behindMSENS_FACTORY=1.
It rendered a tile by executing a validated SELECT against DuckDB and looking the result up against a cell-id COG. Pre-rendered COGs replaced it: nothing reaches it any more, because every metric on every release now has a COG. The two paths were verified equivalent before the switch — 100 of 105 sample tiles byte-identical, the remainder differing by 3–6 pixels out of 262,144 from float32 storage. It remains available for exploratory on-the-fly queries.postgis, tile (
pg_tileserv), tilecache, rest (PostgREST) and swagger — all commented out indocker-compose.ymland not running. PostgreSQL is no longer part of any serving path; the authoritative store is DuckDB, with the published release as partitioned Parquet on S3. Vector geometry is served as static PMTiles by Caddy’sfilesubdomain, and thepgadminentry above is likewise vestigial.