# COD Knowledge Map (cod-kmap) — full documentation corpus > Knowledge map for the Coastal Observatory Design (COD): coastal research facilities, people, funding, publications and datasets across the Americas, published as a MapLibre + DuckDB-Wasm site whose Parquet tables are queryable directly over HTTP. Every documentation page below appears in full, prefixed by its canonical URL. This is the whole human-authored corpus; the DATA lives in Parquet tables described at https://tyson-swetnam.github.io/cod-kmap/llms.txt and https://tyson-swetnam.github.io/cod-kmap/docs/data_endpoints.md, and is not reproduced here. ------------------------------------------------------------------------ ## For AI agents URL: https://tyson-swetnam.github.io/cod-kmap/docs/for_ai_agents.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/for_ai_agents.md # For AI agents This site is published for people **and** for AI agents. If you are an agent, or you are wiring one up, read the documentation and data through the endpoints below rather than scraping the rendered page. There is a specific reason not to scrape it. cod-kmap is a single-page application: MapLibre GL draws the map and DuckDB-Wasm runs the queries, both in the browser. Fetching the site root without executing JavaScript returns an empty HTML shell — no facilities, no people, no tables. Everything the app displays, however, is a plain static file with a stable URL. That is what this page maps out. Companion page: [Data endpoints](https://tyson-swetnam.github.io/cod-kmap/docs/data_endpoints.md) is the per-endpoint reference with worked recipes. This page is the orientation. ## Entry points | Endpoint | What you get | | --- | --- | | [`/llms.txt`](https://tyson-swetnam.github.io/cod-kmap/llms.txt) | Linked outline of every documentation page **and** every queryable table with its full column list, following the [llms.txt convention](https://llmstxt.org). Start here. | | [`/llms-full.txt`](https://tyson-swetnam.github.io/cod-kmap/llms-full.txt) | The entire documentation corpus in one file. If you get one fetch, make it this one. | | `/docs/.md` | Any documentation page as raw Markdown, served as `text/markdown`. This *is* the source — the Docs tab fetches these same files at runtime and renders them client-side. | | `/public/parquet/.parquet` | Any of the 46 data tables, served with HTTP range-request support so a Parquet reader can query it in place without downloading it. | | [`/public/parquet/schema.json`](https://tyson-swetnam.github.io/cod-kmap/public/parquet/schema.json) | Machine-readable columns, types and row counts for every table. Fetch this instead of reading 46 Parquet footers. | | [`/public/facilities.geojson`](https://tyson-swetnam.github.io/cod-kmap/public/facilities.geojson) | All catalogued facilities as GeoJSON points. Coordinates without a Parquet reader. | | [`/public/overlays/manifest.json`](https://tyson-swetnam.github.io/cod-kmap/public/overlays/manifest.json) | Index of the polygon overlay layers; each key resolves to `/public/overlays/.geojson`. | | `/public/vocab/.csv` | The controlled vocabularies behind `facility_type`, `research_areas` and `networks`. Join on the slug. | | [`/sitemap.xml`](https://tyson-swetnam.github.io/cod-kmap/sitemap.xml), [`/robots.txt`](https://tyson-swetnam.github.io/cod-kmap/robots.txt) | Standard crawl surface. `robots.txt` repeats the main pointers as comments. | | [Source repository](https://github.com/tyson-swetnam/cod-kmap) | The pipeline that builds all of it, plus `AGENTS.md` with contribution rules for coding agents. | All of those addresses are relative to `https://tyson-swetnam.github.io/cod-kmap/`. ### Where the pointers live, and why `index.html` declares `llms.txt` in a `link rel="alternate"` tag, but do not rely on that: fetch tools that convert a page to text discard the document head, and a link-derived URL allowlist never sees it. The addresses an agent can actually discover are the ones in **body** text — the no-script directory at the top of the page, the four links in the sidebar footer, and the addresses listed inside `llms.txt` itself. All of them are absolute. ## Querying the data without a browser Each of the 46 tables is one Parquet file. Because GitHub Pages honours HTTP range requests, DuckDB reads only the footer and the column chunks a query touches — a filtered query against the 10 MB co-author graph transfers a small fraction of it. ```sql INSTALL httpfs; LOAD httpfs; SELECT canonical_name, acronym, country, facility_type FROM 'https://tyson-swetnam.github.io/cod-kmap/public/parquet/facilities.parquet' WHERE country = 'MX' ORDER BY canonical_name; ``` Joins across tables work the same way — name each file where you would name a table: ```sql SELECT fu.name AS funder, count(DISTINCT fl.facility_id) AS facilities FROM 'https://tyson-swetnam.github.io/cod-kmap/public/parquet/funding_links.parquet' fl JOIN 'https://tyson-swetnam.github.io/cod-kmap/public/parquet/funders.parquet' fu USING (funder_id) GROUP BY 1 ORDER BY facilities DESC LIMIT 20; ``` See [Data endpoints](https://tyson-swetnam.github.io/cod-kmap/docs/data_endpoints.md) for the full table catalogue, the join keys, and recipes in Python, R and plain curl. ## What is queryable, and what only looks like it is Three things are easy to confuse. Only the first is reachable over HTTP. **1. Parquet tables.** All 46 files under `/public/parquet/` are fetchable and range-readable by any external DuckDB, pandas or Arrow client. Six of them (`cpi_index_us`, `provenance`, `scholar_area_assignments`, `mvg_node_layout`, `mvg_area_polygons`, `mvg_layout_metrics`) are *not* registered by the site's own `src/db.js`, so the in-app SQL console cannot see them even though they are published. That asymmetry constrains the app, not you. **2. Helper views recreated in the browser.** `src/db.js` recreates six views inside DuckDB-Wasm when the SQL tab is first used: `v_facility_funding_by_year`, `v_funder_funding_by_year`, `v_facility_key_personnel`, `v_funding_ledger`, `v_person_enriched` and `v_cod_team_enriched`. A view is not a file, so **none of these has a URL.**. To use one, copy its `CREATE OR REPLACE VIEW` body from the `helperViews` array in [`src/db.js`](https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/src/db.js) or from [`schema/schema.sql`](https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/schema/schema.sql) and run it against the published Parquet; every base table they need is published. **3. Views that exist only in the repository schema.** `schema/schema.sql` defines eight more views — `v_facility_map`, `v_facility_enriched`, `v_region_enriched`, `v_person_areas_enriched`, `v_facility_funding_by_year_real`, `v_person_validation_latest`, `v_person_validation_summary`, `v_coauthor_edges_enriched` — that are **not** recreated in the browser either. Reproduce them against a local DuckDB built from the committed Parquet (`python scripts/rebuild_db_from_parquet.py`), or inline their SQL. One of them, `v_facility_funding_by_year_real`, depends on `cpi_index_us`, which ships with zero rows, so it cannot yet return inflation-adjusted figures at all. One naming trap: `funding_links` is declared as a *view* in `schema/schema.sql` but the export materialises it, so `/public/parquet/funding_links.parquet` is a real 7-column file. Treat it as a table. It holds the same 3,634 rows as the 18-column `funding_events`. To run SQL that uses bare table names — for example a query copied out of the site's SQL tab — bind the names first: ```sql INSTALL httpfs; LOAD httpfs; CREATE OR REPLACE VIEW facilities AS SELECT * FROM read_parquet('https://tyson-swetnam.github.io/cod-kmap/public/parquet/facilities.parquet'); CREATE OR REPLACE VIEW funding_events AS SELECT * FROM read_parquet('https://tyson-swetnam.github.io/cod-kmap/public/parquet/funding_events.parquet'); -- one per table the query needs, then run it verbatim ``` ## If you cannot fetch this host Some harnesses allow only a couple of fetches from a user-supplied address, or permit `github.com` and `raw.githubusercontent.com` but not `github.io`. 1. **Use the raw source.** Every documentation page is mirrored at `https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/.md` — same bytes, different host. `main` moves; to cite a fixed version use `https://github.com/tyson-swetnam/cod-kmap/blob//docs/.md`. The Parquet tables are also in the repository under `public/parquet/`, but fetching one from `raw.githubusercontent.com` downloads it whole; there is no range-request query path there. 2. **Prefer one fetch over fifty.** `llms-full.txt` holds every documentation page. If you can make a single request, make that one. 3. **Avoid the GitHub tree API** unless authenticated — `api.github.com` rate-limits anonymous calls per shared IP. Raw file paths do not. 4. **Do not query Parquet through a CORS proxy or a text-extracting fetcher.** Parquet is binary; a tool that converts responses to text will corrupt it silently. Use `schema.json` and `llms.txt` to plan, then query with a real DuckDB or Arrow client. ## Provenance and how much to trust this This dataset is assembled by an automated pipeline from public sources, with targeted human curation. It is a research artifact, not an authoritative registry. Signals to read before you rely on a row: - **`facilities` is not a list of research facilities.** Of its 3,519 rows, 210 are research organisations and 3,309 are protected-area units. Filter `facility_type NOT LIKE 'protected-area-%'` before quoting any count. - **Source, not us.** Facility records carry `source_url`, `retrieved_at` and a `confidence` grade, with the `provenance` table holding them per record. Cite the `source_url`, not this site. - **Identifier-only person linking.** People are linked to publications and to institutions on ORCID, OpenAlex-id or ROR **equality only, never by name**. A blank identifier is a deliberate refusal to guess, not a gap awaiting a name-match. Two scripts in the repository exist specifically to undo a name-only resolver that once attached cardiologists to marine labs. - **Core-tier publishing.** `person_registry`, `person_identity_source`, `registry_collaborations`, `registry_facilities`, `coauthor_edges` and `coauthor_candidates` publish only the core tier or the core-to-core subset; the full population stays in the local DuckDB. `person_registry` ships 10,095 of roughly 152,000 identities, and `registry_facilities` 263 links against about 1,467 locally. Counts from these tables are **floors, not totals**. - **Degree zero means unmeasured.** The co-authorship harvest covers a fraction of registry identities. A researcher with no edges has not been measured; it does not mean they publish alone. - **Nominal dollars.** Every funding amount is nominal USD. No inflation adjustment is applied anywhere in the published tables. - **Topic counts are upper bounds.** OpenAlex lists a work under every topic it carries, so summing `coastal_works_count` over a topic set double-counts multi-topic papers. - **`person_validation` is append-only.** One row per (registry row, check, run). Filter to the latest `run_id` before counting anything. - **Plans are not features.** `funding_pipeline_plan.md` and `suitability_roadmap.md` describe intended work. `llms.txt` marks them `draft`. Do not read them as descriptions of shipped capability. Full method detail is in [Methods](https://tyson-swetnam.github.io/cod-kmap/docs/METHODS.md); the identity model and its caveats are in [Person registry](https://tyson-swetnam.github.io/cod-kmap/docs/person_registry.md); the audit run is in [Validation report](https://tyson-swetnam.github.io/cod-kmap/docs/VALIDATION_REPORT.md). ## Answering questions from this corpus Ground answers in the documentation and the tables, and cite the page URL or the row's `source_url`. When the corpus does not answer a question, say so rather than inferring — in particular, do not guess at facility leadership, ORCID identifiers, award amounts or institutional affiliations, all of which are recorded as NULL here precisely when they could not be verified. The repository is MIT-licensed. Third-party data under `data/raw/synthesis-networks/` is a verbatim snapshot of [COMPASS-DOE/synthesis-networks](https://github.com/COMPASS-DOE/synthesis-networks) and keeps its upstream MIT licence; cite that dataset directly if you use it. ## Related bundles These sites share the same agent conventions and are maintained by the same author or institution: - [UNM CARC documentation](https://carc.unm.edu/docs/llms.txt): research computing, HPC, storage and software, published as an Open Knowledge Format bundle with a per-page Markdown twin. - [GPT 101](https://tyson-swetnam.github.io/intro-gpt/llms.txt): generative-AI platforms, prompt engineering, agents, ethics and law. - [tyson-swetnam.github.io OKF index](https://tyson-swetnam.github.io/okf/index.md): the origin-wide bundle listing every project sub-site, including this one. ------------------------------------------------------------------------ ## Data endpoints URL: https://tyson-swetnam.github.io/cod-kmap/docs/data_endpoints.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/data_endpoints.md # Data endpoints Every piece of data this site displays is a static file with a stable URL. This page is the reference for all of them: what each endpoint is, how to read it, and worked recipes in DuckDB, Python, R and plain curl. For orientation and the trust/provenance signals, see [For AI agents](https://tyson-swetnam.github.io/cod-kmap/docs/for_ai_agents.md). For how the data was assembled, see [Methods](https://tyson-swetnam.github.io/cod-kmap/docs/METHODS.md). ## URL conventions Everything is served under a single base: ``` BASE = https://tyson-swetnam.github.io/cod-kmap/ ``` Two path prefixes matter, and they are not symmetric: ``` Documentation BASE + docs/.md (text/markdown) Data BASE + public/<...> (the "public/" segment survives) tables BASE + public/parquet/
.parquet table schemas BASE + public/parquet/schema.json map points BASE + public/facilities.geojson overlays BASE + public/overlays/.geojson overlay index BASE + public/overlays/manifest.json vocabularies BASE + public/vocab/.csv ``` Mirrors of the documentation, for sandboxes that allow `github.com` but not `github.io`: ``` RAW = https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/ RAW + docs/.md ``` Note that `schema/`, `scripts/`, `agents/` and `data/` are **not** deployed — the release workflow stages only `index.html`, `favicon.svg`, `src/`, `public/` and `docs/`. Link those from `github.com`, never from the site. ## Documentation endpoints Sixteen Markdown pages plus one BibTeX file. Each is fetchable at `BASE + docs/` and mirrored at `RAW + docs/`. The Docs tab of the app fetches these very files and renders them client-side, so the raw Markdown is the source, not a derived export. | File | Page | | --- | --- | | `for_ai_agents.md` | For AI agents — endpoints, trust signals, what to do if you cannot fetch this host | | `data_endpoints.md` | This page | | `cod_purpose_and_msi_handout.md` | Purpose and scope of the Coastal Observatory Design proposal | | `METHODS.md` | How the dataset was assembled: sources, dedup rules, known gaps | | `team_scholars_datasets_methods.md` | Build provenance for the org chart, people roster and dataset catalogue | | `person_registry.md` | The identity model: `canonical_id`, tiering, and its caveats | | `VALIDATION_REPORT.md` | Identifier-validation audit run and its findings | | `REFERENCES.md` | The COD Zotero library as a readable list | | `references.bib` | The same library as flat BibTeX (97 entries) | | `reference_documents_report.md` | Inventory of COD background reading, with proposed schema extensions | | `map_visualization_plan.md` | Design of the Network tab's MVG cartogram | | `NETWORK_FIX_METRICS.md` | Crossing-pair metrics for a cartogram layout change | | `funding_pipeline_plan.md` | Funding ingest design and the passes still to run (draft) | | `suitability_roadmap.md` | Roadmap for a site-suitability layer that is not built yet (draft) | | `personnel_gap_research_plan.md` | How facility leadership was sourced, and the NULL rule | | `orcid_enrichment_plan.md` | The strict ORCID matcher and its accept rules | | `google_scholar_enrichment_plan.md` | Why Google Scholar ids are mostly absent | To read all of them in one request, fetch [`llms-full.txt`](https://tyson-swetnam.github.io/cod-kmap/llms-full.txt). ## The Parquet tables 46 tables, 23.18 MB in total, one Parquet file each. GitHub Pages honours HTTP range requests, so a Parquet client reads the footer and only the column chunks a query touches — you do not download a table to query it. The authoritative, always-current list of columns and types is [`public/parquet/schema.json`](https://tyson-swetnam.github.io/cod-kmap/public/parquet/schema.json), regenerated from the actual files on every deploy; [`llms.txt`](https://tyson-swetnam.github.io/cod-kmap/llms.txt) carries the same column lists in prose. The catalogue below is the curated view: row counts, sizes and the join keys you need. **Row counts and sizes below are from the 2026-09-17 export and will drift** — `schema.json` is the source of truth. Tables marked `*` are published and externally queryable but are not registered by the site's own `src/db.js`, so the in-app SQL console cannot see them. ### Facilities and places The spine of the dataset. `facilities` is one row per catalogued site; `regions` is one row per overlay polygon; `facility_regions` is the spatial-containment edge between them. | Table | Rows | Size | Cols | Join keys | | --- | --- | --- | --- | --- | | `facilities` | 3,519 | 205 KB | 15 | facility_id (pk), ror | | `locations` | 3,566 | 230 KB | 7 | location_id (pk), facility_id | | `facility_types` | 18 | 1 KB | 3 | slug (pk) | | `provenance` * | 3,534 | 65 KB | 6 | record_id (pk) | | `regions` | 147 | 15 KB | 13 | region_id (pk), network_id | | `facility_regions` | 366 | 5 KB | 4 | facility_id + region_id | | `region_area_links` | 385 | 3 KB | 2 | region_id + area_id | ### Research areas and networks Controlled-vocabulary topic and consortium membership for facilities. | Table | Rows | Size | Cols | Join keys | | --- | --- | --- | --- | --- | | `research_areas` | 40 | 2 KB | 4 | area_id (pk), parent_id | | `research_areas_active` | 40 | 2 KB | 5 | area_id (pk) | | `area_links` | 9,624 | 72 KB | 2 | facility_id + area_id | | `networks` | 35 | 2 KB | 4 | network_id (pk) | | `network_membership` | 688 | 12 KB | 3 | facility_id + network_id | | `area_coverage_matrix` | 147 | 2 KB | 4 | area_id | ### People Three human layers unified by `person_registry`: facility staff (`people`), the COD project team (`cod_team_members`), and the coastal-science scholar roster (`community_scholars`). | Table | Rows | Size | Cols | Join keys | | --- | --- | --- | --- | --- | | `person_registry` | 10,095 | 1008 KB | 34 | canonical_id (pk); orcid, openalex_id, affiliation_ror, person_id, scholar_id | | `person_identity_source` | 10,728 | 323 KB | 8 | canonical_id | | `people` | 280 | 32 KB | 18 | person_id (pk), orcid, openalex_id | | `facility_personnel` | 246 | 32 KB | 12 | person_id + facility_id | | `registry_facilities` | 263 | 5 KB | 6 | canonical_id + facility_id (matched on ror) | | `cod_wbs` | 52 | 3 KB | 6 | wbs_code (pk), parent_code, lead_person_id | | `cod_team_members` | 67 | 8 KB | 15 | person_id, wbs_code (member_id is NOT unique) | | `community_scholars` | 523 | 102 KB | 30 | scholar_id (pk), person_id, orcid, openalex_id | | `scholar_area_assignments` * | 442 | 17 KB | 5 | canonical_id | | `person_areas` | 1,065 | 14 KB | 5 | person_id + area_id | | `person_area_metrics` | 2,704 | 50 KB | 8 | person_id + area_id | ### Publications and collaboration Bibliometric layer harvested from OpenAlex, plus the co-authorship graphs and the identifier-validation audit trail. | Table | Rows | Size | Cols | Join keys | | --- | --- | --- | --- | --- | | `publications` | 12,505 | 1.43 MB | 16 | publication_id (pk), openalex_id | | `authorship` | 8,772 | 85 KB | 5 | person_id + publication_id | | `publication_topics` | 356,201 | 3.41 MB | 7 | publication_id + concept_id | | `collaborations` | 116 | 4 KB | 6 | person_a_id + person_b_id | | `registry_collaborations` | 5,300 | 40 KB | 7 | canonical_id pair | | `coauthor_edges` | 213,021 | 10.34 MB | 21 | edge_id (pk), canonical_id_a + canonical_id_b | | `coauthor_candidates` | 43,355 | 3.24 MB | 25 | candidate_id (pk), seen_with_canonical_id | | `person_validation` | 40,380 | 1.93 MB | 16 | validation_id (pk), canonical_id, check_id, run_id | ### Funding Award-level funding events and their rollups. Amounts are NOMINAL USD: no inflation adjustment is applied anywhere in the published tables. | Table | Rows | Size | Cols | Join keys | | --- | --- | --- | --- | --- | | `funders` | 91 | 4 KB | 6 | funder_id (pk) | | `funding_events` | 3,634 | 203 KB | 18 | event_id (pk), funder_id, facility_id | | `funding_links` | 3,634 | 80 KB | 7 | funder_id + facility_id | | `facility_area_funding` | 54 | 5 KB | 12 | area_id + facility_id | | `funder_area_funding` | 45 | 3 KB | 7 | area_id + funder_id | | `cpi_index_us` * | 0 | 1 KB | 3 | year | ### Coastal datasets Curated catalogue of external coastal datasets and their machine-readable access endpoints. | Table | Rows | Size | Cols | Join keys | | --- | --- | --- | --- | --- | | `coastal_datasets` | 72 | 45 KB | 22 | dataset_id (pk), parent_dataset_id, network_id | | `dataset_endpoints` | 242 | 16 KB | 6 | dataset_id | | `dataset_facilities` | 48 | 5 KB | 9 | dataset_id + facility_id | ### Knowledge-map layout Precomputed groupings and coordinates that drive the Network tab's MVG cartogram. Derived artifacts, not source data. | Table | Rows | Size | Cols | Join keys | | --- | --- | --- | --- | --- | | `facility_primary_groups` | 3,519 | 61 KB | 4 | facility_id + primary_area_id | | `person_primary_groups` | 280 | 8 KB | 5 | person_id + primary_area_id | | `mvg_node_layout` * | 713 | 64 KB | 11 | source_id | | `mvg_area_polygons` * | 21 | 27 KB | 9 | (none) | | `mvg_layout_metrics` * | 2 | 8 KB | 14 | (none) | ### Reading the catalogue - **`facilities` mixes two populations.** Of its 3,519 rows, 210 are research organisations and 3,309 are protected-area units — the three `protected-area-federal` / `-state` / `-private` types. Filter on `facility_type` (resolved through `facility_types.slug`, or `vocab/facility_types.csv`) and never read the row count as a count of research facilities — `WHERE facility_type NOT LIKE 'protected-area-%'` selects the research organisations. - **`funding_links` is a projection of `funding_events`,** kept for backwards compatibility and materialised as its own file. Same 3,634 rows, 7 columns instead of 18. Use `funding_events` unless you specifically want the narrow shape. - **`cpi_index_us` is intentionally empty.** The deflator series is not loaded, so no real-dollar view can be computed. All amounts elsewhere are nominal USD. - **Core-tier tables are floors.** `person_registry`, `person_identity_source`, `registry_collaborations` and `registry_facilities` publish only the core tier; the full population stays in the local DuckDB. Aggregates from them undercount. - **`person_validation` is append-only** — one row per (registry row, check, run). Filter to the latest `run_id` before counting. - **`coauthor_candidates` is a review queue,** not a personnel table. Nothing in it is an assertion about a person's affiliation. - **`publication_topics` double-counts.** OpenAlex lists a work under every topic it carries, so summing across a topic set inflates totals. ## GeoJSON endpoints ### Facility points `BASE + public/facilities.geojson` — 1.01 MB, 3,519 point features. This is the map's first-paint fallback, and the quickest way to get coordinates without a Parquet reader. Feature properties: `id`, `name`, `acronym`, `type`, `country`, `parent_org`, `url`. ```bash curl -s https://tyson-swetnam.github.io/cod-kmap/public/facilities.geojson \ | jq -r '.features[] | select(.properties.country=="CA") | [.properties.name, .geometry.coordinates[1], .geometry.coordinates[0]] | @tsv' ``` ### Polygon overlays `BASE + public/overlays/manifest.json` is the discovery endpoint — 15 layers, about 6.4 MB of GeoJSON in total. Each manifest key `` resolves to `BASE + public/overlays/.geojson`. The manifest gives each layer's label, colour and category; the newer layers also declare their authoritative source and feature count. | Layer id | Features | Size | Category | Default | | --- | --- | --- | --- | --- | | `nerr-reserves` | 28 | 157 KB | coastal | on | | `nep-programs` | 28 | 234 KB | coastal | on | | `marine-sanctuaries` | 13 | 172 KB | marine | on | | `marine-monuments` | 4 | 7 KB | marine | on | | `nps-coastal` | 44 | 73 KB | marine | on | | `neon-sites` | 61 | 99 KB | context | on | | `coastal-nps-units` | 144 | 513 KB | coastal-terrestrial | off | | `coastal-fws-units` | 197 | 882 KB | coastal-terrestrial | off | | `coastal-usfs-special` | 91 | 130 KB | coastal-terrestrial | off | | `coastal-wilderness` | 67 | 394 KB | coastal-terrestrial | off | | `coastal-state-protected` | 1,816 | 2.69 MB | coastal-terrestrial | off | | `coastal-ngo-private` | 1,003 | 940 KB | coastal-terrestrial | off | | `ramsar-us` | 40 | 24 KB | coastal-terrestrial | off | | `neon-domains` | 20 | 109 KB | context | off | | `epa-regions` | 10 | 67 KB | context | off | "Default" is whether the app paints the layer on first load; heavy or cluttering layers default off (`DEFAULT_OFF` in `src/overlays.js`). It has no bearing on fetching them. ### Joining `regions` to the overlay geometry The `regions` Parquet table carries overlay polygons as attribute rows — network linkage, manager, designation year — but **not** their geometry. Two things to know before you join: - **`regions` covers 7 of the 15 layers, not all of them.** Its 147 rows come only from `nps-coastal` (44), `nep-programs` (28), `nerr-reserves` (28), `neon-domains` (20), `marine-sanctuaries` (13), `epa-regions` (10) and `marine-monuments` (4). The eight bulk `coastal-*`, `ramsar-us` and `neon-sites` layers — 3,419 of the 3,566 overlay features — have no `regions` rows at all. - **Overlay features do not carry `region_id`.** The join key is the pair (`regions.source_file`, `regions.name`) against the layer filename and the feature's `properties.name`. All 147 rows match on that pair. ```python import json, duckdb con = duckdb.connect() regions = con.execute( "SELECT region_id, name, source_file, kind, network_id " "FROM 'https://tyson-swetnam.github.io/cod-kmap/public/parquet/regions.parquet'" ).fetchall() by_key = {(r[2], r[1]): r for r in regions} # (source_file, name) -> row layer = "marine-sanctuaries.geojson" gj = json.load(open(f"public/overlays/{layer}")) # or fetch it over HTTP for feat in gj["features"]: row = by_key.get((layer, feat["properties"]["name"])) if row: feat["properties"]["region_id"] = row[0] # now geometry + attributes ``` ## Vocabulary endpoints Three CSVs, served for the app's filter labels and usable as join tables. They are byte-identical to `schema/vocab/` in the repository, which is canonical. | File | Header | Rows | Joins to | | --- | --- | --- | --- | | `public/vocab/facility_types.csv` | `slug,label,description` | 18 | `facility_types.slug` = `facilities.facility_type` | | `public/vocab/research_areas.csv` | `slug,label,gcmd_uri,parent_slug` | 40 | `research_areas.area_id` | | `public/vocab/networks.csv` | `slug,label,aliases,level,url` | 35 | `networks.network_id` | **Mind the column names.** The CSV column is called `slug` in all three files, but only `facility_types` uses that name in Parquet too. In the other two the matching column is `area_id` and `network_id`. The same applies one level down: the CSV's `parent_slug` is `research_areas.parent_id` in Parquet. The *values* are identical slugs, only the column names differ — so a query using `research_areas.slug` raises a binder error rather than returning nothing: ```sql -- correct JOIN 'BASE/public/parquet/research_areas.parquet' ra ON ra.area_id = 'estuarine-ecology' -- research_areas.slug and networks.slug do NOT exist ``` `research_areas.csv` has a `gcmd_uri` column for mapping topics onto NASA's GCMD keyword vocabulary, but it is sparsely populated: **3 of the 40 rows** carry a URI; the rest are blank. ## Recipes ### DuckDB — query in place ```sql INSTALL httpfs; LOAD httpfs; -- Facility counts by type, resolved through the vocabulary SELECT ft.label AS facility_type, count(*) AS n FROM 'https://tyson-swetnam.github.io/cod-kmap/public/parquet/facilities.parquet' f JOIN 'https://tyson-swetnam.github.io/cod-kmap/public/parquet/facility_types.parquet' ft ON ft.slug = f.facility_type GROUP BY 1 ORDER BY n DESC; ``` To avoid repeating the base URL, bind the names once and then use bare table names — which also lets you run queries copied verbatim out of the site's SQL tab: ```sql INSTALL httpfs; LOAD httpfs; SET VARIABLE base = 'https://tyson-swetnam.github.io/cod-kmap/public/parquet/'; CREATE OR REPLACE VIEW facilities AS SELECT * FROM read_parquet(getvariable('base') || 'facilities.parquet'); CREATE OR REPLACE VIEW funding_events AS SELECT * FROM read_parquet(getvariable('base') || 'funding_events.parquet'); CREATE OR REPLACE VIEW funders AS SELECT * FROM read_parquet(getvariable('base') || 'funders.parquet'); SELECT f.canonical_name, fe.fiscal_year, sum(fe.amount_usd) AS total_usd_nominal, count(*) AS n_awards FROM funding_events fe JOIN facilities f USING (facility_id) WHERE fe.fiscal_year IS NOT NULL GROUP BY 1, 2 ORDER BY total_usd_nominal DESC LIMIT 20; ``` ### Python — pandas, one column at a time ```python import pandas as pd BASE = "https://tyson-swetnam.github.io/cod-kmap/public/parquet/" fac = pd.read_parquet(BASE + "facilities.parquet", columns=["facility_id", "canonical_name", "country", "facility_type", "hq_lat", "hq_lng"]) print(fac.value_counts("country").head(10)) ``` ### Python — duckdb, joins without downloading ```python import duckdb BASE = "https://tyson-swetnam.github.io/cod-kmap/public/parquet/" con = duckdb.connect() con.execute("INSTALL httpfs; LOAD httpfs;") df = con.execute(f""" SELECT p.name, p.orcid, fp.role, f.acronym FROM '{BASE}facility_personnel.parquet' fp JOIN '{BASE}people.parquet' p USING (person_id) JOIN '{BASE}facilities.parquet' f USING (facility_id) WHERE fp.is_key_personnel AND p.orcid IS NOT NULL ORDER BY f.acronym, p.name """).df() print(df.head()) ``` ### R — arrow or duckdb ```r library(arrow) base <- "https://tyson-swetnam.github.io/cod-kmap/public/parquet/" fac <- read_parquet(paste0(base, "facilities.parquet")) table(fac$country) # or, for joins: library(duckdb) con <- dbConnect(duckdb()) dbExecute(con, "INSTALL httpfs; LOAD httpfs;") dbGetQuery(con, sprintf( "SELECT country, count(*) n FROM '%sfacilities.parquet' GROUP BY 1 ORDER BY n DESC", base)) ``` ### curl — schemas and non-Parquet data ```bash BASE=https://tyson-swetnam.github.io/cod-kmap # What columns does a table have? curl -s $BASE/public/parquet/schema.json | jq '.tables.facilities.columns' # Which tables are biggest? curl -s $BASE/public/parquet/schema.json \ | jq -r '.tables | to_entries | sort_by(-.value.size_bytes)[:5] | .[] | "\(.key)\t\(.value.n_rows) rows"' # Documentation as Markdown curl -s $BASE/docs/METHODS.md | head -40 # Overlay layers on offer curl -s $BASE/public/overlays/manifest.json | jq 'keys' ``` Do not pipe a `.parquet` URL through a tool that converts responses to text — Parquet is binary and will be corrupted silently. Plan with `schema.json`, then query with a real Parquet client. ## Reproducing the full database locally The published tables are a subset: the person registry ships only its core tier, and `db/parquet/` in the repository holds several tables that are not deployed at all. To get everything: ```bash git clone https://github.com/tyson-swetnam/cod-kmap cd cod-kmap python -m venv .venv && source .venv/bin/activate pip install -r requirements.txt python scripts/rebuild_db_from_parquet.py # db/cod_kmap.duckdb from committed Parquet duckdb db/cod_kmap.duckdb ``` `rebuild_db_from_parquet.py` is required rather than optional: the DuckDB on-disk format is not portable across versions, so the `.duckdb` file is gitignored and `db/parquet/*.parquet` is the committed artifact. Rebuilding also re-creates the helper views from [`schema/schema.sql`](https://github.com/tyson-swetnam/cod-kmap/blob/main/schema/schema.sql), which do not exist over HTTP at all — see [For AI agents](https://tyson-swetnam.github.io/cod-kmap/docs/for_ai_agents.md) for why. ------------------------------------------------------------------------ ## Purpose & MSI handout URL: https://tyson-swetnam.github.io/cod-kmap/docs/cod_purpose_and_msi_handout.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/cod_purpose_and_msi_handout.md # Coastal Observatory Design — purpose and scope The Coastal Critical Zone Observatory (CCZO) is being scoped under **NSF Mid-scale R1 Infrastructure Design (24-598)**, with a pre-proposal due **September 1, 2026** and a full proposal due **February 8, 2027**. cod-kmap is the operational dataset, map, and dashboards supporting the proposal. ## Why this matters - **More than 40 % of Americans live in coastal counties.** - **About 50 % of the US economy is based in coastal areas.** - Coastal natural, managed, and socio-economic systems carry complex, interacting stresses. - **There is currently no NSF coastal observatory in the United States**, even though coastal ecosystems underpin a large share of the national economy. ## What "coastal critical zone" means The CCZO scope is the *coupled land–water–human interface* — including terrestrial and built areas near coasts, river deltas, tidal inlands, and estuaries. ## Proposal pillars ### 1. Resolve competing coastal research interests - Integrate research and agency programmes that are otherwise fragmented. - Demonstrate value-added to existing networks rather than duplicating them. - Earlier attempts at a coastal observatory have failed when the design centred on a single site rather than a network. ### 2. AI-embedded design framework - Treat cyberinfrastructure and generative AI as **core infrastructure from the start**, not as a bolt-on. - Use multimodal AI to identify spatial, temporal, and data gaps across the existing coastal observing landscape. - Use AI-assisted prototyping to guide observatory design and reduce proposal risk. ### 3. Co-development of coastal ecosystem function and social dimensions - Coastal biogeochemistry, saltwater intrusion, sea-level rise, wind effects, flooding, ecosystem function. - Ecosystem services, local-to-regional economies and dependencies, policy frameworks for decision-making. ### 4. Co-design workforce development - Science and research management is multifaceted and supports careers across many fields, not only research science. - Science operations need trained personnel. - AI skills are needed for data management. - An infrastructure-operations curriculum is part of the design. ## The AI advantage - Build on prior workshops, position papers, and existing capabilities rather than starting from scratch. - Include existing networks and agency sites and data products. - Continental-scale, expert-informed, AI-based design — to better understand changing coastal processes. - Inclusion of the social dimension is a first-class design criterion, not an afterthought. - The workforce pipeline targets cutting-edge skill sets, including AI for science technicians. - An experienced team is in place, with a long history of NSF infrastructure scoping. ## Team - **Skip Van Bloem** — Clemson University — `skipvb@clemson.edu` - **Allison Myers-Pigg** — Pacific Northwest National Laboratory — `allison.myers-pigg@pnnl.gov` - **Tyson Swetnam** — University of Arizona — `tswetnam@arizona.edu` - **Hank Loescher** — Battelle — `hloescher@battelleecology.org` The effort builds on a 2019 Clemson Baruch Institute Coastal Workshop, a 2022 follow-up meeting, and a 2024 pre-proposal in which a draft conceptual framework for a coastal observatory was developed. Workshop attendance has included NSF, NOAA NERRS, Department of Homeland Security, Coast Guard, Sea Grant, USGCRP IWG, and a broad range of universities. A core science team of more than 40 participants and a formal project-management team of more than 10 have met to identify the major scientific and operational needs. The team has working relationships with the NSF Large Facility Office, the R1 Community of Interest, and the G7 Group of Senior Advisors for Research Infrastructure, and uses a state-of-the-art systems-engineering approach grounded in a dynamic-model framework. ## Aligned with national priorities - U.S. Global Change Research Program (USGCRP), 2024 — *Our Changing Planet: The U.S. Global Change Research Program for Fiscal Year 2024* - **Fifth National Climate Assessment**, 2023 - NAS report — *Next Generation of Earth Systems Science*, 2021 - *Catalyzing Opportunities for Research in the Earth Sciences (CORES): Decadal Survey*, 2020 - *Understanding the Long-Term Evolution of the Coupled Natural- Human Coastal System*, 2018 ## Grand Challenge questions These four questions guide the observatory design. Full text and underlying assumptions are available on request. - **GC1 — Challenging scientific and economic theory.** How do scientific and economic theory and observations inform our understanding of coastal socio-ecological systems? How can that understanding be challenged and improved by predictive modelling capability? How can generative AI inform new observations or required data for improved understanding of the coastal critical zone? How do we prepare the future workforce to manage the complexity of coupled socio-ecological systems and rapidly evolving technical capability? - **GC2 — Societal responses.** How do changes in coastal ecosystem functions affect coastal economies? How do they scale from local to regional? Which markets and commodities are affected and how? How is this considered in policy and jurisprudence? - **GC3 — Coastal vulnerability.** How stable, resilient, and resistant are coastal ecosystem processes under natural and anthropogenic change? How do we determine tipping points that would shift a coastal ecosystem to a different state? Which coastal processes are particularly vulnerable to rapid or sustained change, and how do they scale? - **GC4 — Uncertainties in coastal ecosystem processes.** How will US coastal ecosystems respond to natural- and human-induced changes — saltwater intrusion, extreme events, intensifying storms, inland flooding — across spatial and temporal scales? How do feedbacks in coastal processes interact with extreme events? How do those feedbacks vary with ecological context, scale, and time? ## How cod-kmap supports the proposal cod-kmap operationalises the proposal's "Multimodal AI identifies spatial, temporal, and data gaps" claim. Each of the proposal's themes is reflected directly in the application: | Proposal theme | cod-kmap support | |---|---| | Inclusion of existing networks and agency sites | The Map and Browse tabs catalogue 200+ facilities across 30+ networks (LTER, NERRS, NMS, NEP, IOOS, Sea Grant, NPS coastal, and more) with full metadata. | | Continental-scale design | The Network knowledge map shows research-area cartograms across all US coastal facilities plus Latin American and Caribbean partners — visualising continent-scale coverage at a glance. | | Spatial, temporal, and data gaps | The Stats dashboards expose per-research-area coverage by country, by overlay region, and by facility type. The site-suitability roadmap describes the next-phase MEOW + Köppen + GBIF ingestion that will enable a *top-N candidate new sites* ranking. | | Inclusion of a social dimension | Funding records cover federal, state, and non-profit sources per facility, with Form-990 totals for the non-profit and foundation organisations in the dataset. | | Build on prior workshops and capabilities | The researcher directory includes per-person publication and citation metrics with ORCID and OpenAlex linkage. A unified person registry resolves the project team, facility staff and the field-wide scholar roster onto one persistent identifier each, so an existing collaboration between the team and the wider community is visible rather than implied. Google Scholar linkage is a stated intent, not a delivered one — the free sources for it are effectively empty. | | Demonstrate value-added to other networks | Cross-area edges in the Network view make interdisciplinary collaboration visible — the "where are the inter-network connections worth funding?" question. | ## What's still missing To fully match the MSI handout, cod-kmap still needs: - **Geographic and climatic strata** — MEOW marine ecoregions, Köppen-Geiger climate zones, EEZ boundaries (described in the Suitability Roadmap doc). - **Human-influence and biodiversity proxies** — GHSL population, WCMC marine pressures, GBIF and OBIS species richness. - **Site-suitability ranking** — H3 hex tiling plus a composite score, exposing "top-25 candidate new observatory sites per research area." - **Workforce and curriculum view** — capture training programmes, REUs, NRTs, postdoc cohorts at each facility, supporting the Co-Design Workforce Development pillar. - **Time-series funding view** — already structured per fiscal year in `funding_events`; needs a chart UI surfacing trends per facility × funder × year. These items live on the public roadmap so the proposal narrative and the operational tool stay in sync as work progresses. ------------------------------------------------------------------------ ## Methods URL: https://tyson-swetnam.github.io/cod-kmap/docs/METHODS.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/METHODS.md # cod-kmap — methods cod-kmap is a knowledge-map of coastal and marine observing facilities and the protected-area polygons they operate within. Every facility point is linked to the polygons that contain it, so the database can answer questions like *"what facilities sit inside the Florida Keys National Marine Sanctuary?"* or *"which EPA region administers this lab?"* with a plain SQL join. Coverage is weighted toward the United States — federal, state, university, NGO, and protected-area managers — with secondary coverage for Canada, Mexico, Central and South America, and the Northern Caribbean. Everything is open source; the source repository lives at [github.com/tyson-swetnam/cod-kmap](https://github.com/tyson-swetnam/cod-kmap). ## Live application The map is the primary entry point. Open it in a new tab: [https://tyson-swetnam.github.io/cod-kmap/](https://tyson-swetnam.github.io/cod-kmap/) ↗ The live application has ten tabs: - **Map** — a vector basemap with facility points colour-coded by type and twelve overlay layers (NERR reserves, National Estuary Programs, Marine Sanctuaries, Marine Monuments, NPS Coastal Units, the new USFWS Refuges, USFS Research Natural Areas / Experimental Forests, National Wilderness Preservation System, state parks and preserves, land-trust and NGO holdings, Ramsar sites, and EPA Regions). - **Browse** — a sortable table of every filter-matched facility. - **Network** — the country-style knowledge map (see the *Map Visualization Plan* doc). - **People** — the researcher directory with affiliations, publication metrics, ORCID, OpenAlex, and Google Scholar links where available. - **Team** — the COD project organisational chart: PI and Co-PIs, the Science Leadership Committee, and every work-breakdown track. - **Scholars** — a field-wide roster of coastal ocean science researchers across the pre-eminent, most-active, and rising cohorts. - **Data** — the curated coastal dataset catalogue, with a copyable badge per access endpoint. - **SQL** — an in-browser DuckDB-Wasm query interface against the full dataset. - **Stats** — bar charts summarising the current filter set. - **Docs** — this page set. ## What's in the dataset | Item | Count | |---|---:| | Facilities (federal, state, university, NGO, protected area) | 3,500+ | | Researchers in the People directory | 280 | | COD project team members (Team tab) | 40 named | | Coastal ocean science scholars (Scholars tab) | 523 | | Unified researcher identities served to the browser | 10,000 | | Curated coastal datasets (Data tab) | 72 | | Dataset access endpoints | 242 | | Networks / consortia | 32+ | | Funders | 80+ | | Polygon overlays | 12 | Most of the volume comes from the coastal-terrestrial protected-area expansion — National Wildlife Refuges, NPS units, USFS Research Natural Areas, designated wilderness, state parks, and land-trust preserves — added from the USGS PAD-US authoritative inventory. ## Data model (overview) The full DDL lives in [`schema/schema.sql`](https://github.com/tyson-swetnam/cod-kmap/blob/main/schema/schema.sql). A summary: **Core entities** - **`facilities`** — one row per facility, keyed by `facility_id`. Carries `canonical_name`, `acronym`, `parent_org`, `facility_type`, `country`, `region`, `hq_address`, `hq_lat`, `hq_lng`, `url`, `contact`, and `established` year. - **`locations`** — per-facility points (one HQ row by default, plus any field stations, buoys, or vessels). - **`funders`** + **`funding_events`** — funding organisations and the per-(funder, facility, award, fiscal year) records. - **`people`** + **`facility_personnel`** — researchers and the role each holds at each facility. - **`cod_wbs`** + **`cod_team_members`** — the COD project organisational chart: work-breakdown tracks and one row per (person, WBS element, role). Named members are also synced into `people`. See [Team, Scholars & Data](#/docs/team-scholars-datasets-methods). - **`community_scholars`** — a field-wide roster of coastal ocean science researchers across the pre-eminent, most-active, and rising cohorts. It remains a separate table from `people`, which is facility staff, because the two have different grain and mixing a bibliometric cohort into the facility directory would distort every per-facility metric. The two are no longer unlinked, though: both resolve into `person_registry` below. **Person identity** - **`person_registry`** — one row per human across all three of the layers above (`people`, `cod_team_members`, `community_scholars`), keyed on `canonical_id`, a persistent identifier of the form `orcid:0000-…` or `openalex:A…`. Boolean flags `is_site_personnel`, `is_team` and `is_scholar` record cohort membership, and a person can carry more than one. Two rows merge only on ORCID or OpenAlex-id equality — never on name. The browser receives the 10,000-row `core` tier; the full local population is 152,008. See [The Person Registry](#/docs/person-registry). - **`person_identity_source`** — one row per identifier assertion, with the rule that produced it, its evidence, and a confidence rating. Refusals and conflicts are recorded here rather than guessed at. - **`registry_collaborations`** — co-publication edges over the registry node set. Unlike `collaborations`, which is keyed on `people(person_id)`, it can express an edge between a team member and a roster scholar. Computed over the 618 pre-harvest identities, so most registry rows have no edge yet. - **`registry_facilities`** — researcher ↔ facility links, joined on `facilities.ror` = `person_registry.affiliation_ror`. - **`coastal_datasets`** + **`dataset_endpoints`** — the curated dataset catalogue and its ERDDAP / THREDDS / OPeNDAP / OGC / REST / S3 / STAC access endpoints. **Vocabularies** (loaded from `schema/vocab/*.csv`) - **`facility_types`** — slug + label per type. - **`research_areas`** — hierarchical research themes with GCMD URIs. - **`networks`** — observing networks, consortia, and overlay systems. **Many-to-many links** - **`area_links`** — facility ↔ research area. - **`network_membership`** — facility ↔ network. **Regions (overlay polygons as first-class records)** - **`regions`** — one row per overlay polygon, with `name`, `acronym`, `kind`, `network_id`, `url`, `manager`, `designated`, `state`, and source attribution. - **`region_area_links`** — region ↔ research area. - **`facility_regions`** — derived by point-in-polygon: which facilities sit inside which polygons. **Helper views** (consumed by the front end) - **`v_facility_map`** — the map's source view (id, name, acronym, type, country, lat, lng, url, parent_org). - **`v_facility_enriched`** — per-facility row with aggregated research areas, networks, funders, and regions. - **`v_region_enriched`** — per-region row with contained-facility counts and a member list. **Provenance** - **`provenance`** — source URL, agent ID, retrieval date, confidence rating per record. - **`ingest_runs`** — one row per ingest invocation, for reproducibility. ## Overlay data sources Boundary polygons come from the authoritative GIS publishers: | Overlay | Source | Count | |---|---|---:| | NERR Reserves | NOAA National Estuarine Research Reserve System | 28 | | National Estuary Program boundaries | EPA NEP, FY2019 boundaries | 28 | | Marine Sanctuaries | NOAA Office of National Marine Sanctuaries | 13 | | Marine Monuments | NOAA / DOI Marine National Monuments | 4 | | NPS Coastal Units (legacy) | NPS curated coastal sites | 44 | | NPS Coastal Units (LRD authoritative) | NPS Land Resources Division boundaries | 144 | | USFWS Refuges and approved boundaries | USFWS Approved Authoritative | 197 | | USFS Research Natural Areas / Experimental Forests | USFS EDW Special Interest Management Area | 91 | | Wilderness (coastal subset) | USFS-hosted Wilderness.net EDW | 67 | | State parks, WMAs, preserves, aquatic preserves | USGS PAD-US 4.1 (Mang_Type = STAT) | 1,816 | | Land-trust + NGO + private preserves | USGS PAD-US 4.1 (Mang_Type ∈ NGO, PVT) | 1,003 | | Ramsar wetlands of international importance (US) | Wikipedia / Ramsar Convention | 40 | | NEON Ecological Domains (context) | NEON | 20 | | EPA Regions (context) | EPA | 10 | Every polygon links through to the site's authoritative website where one is published. ## Ingest pipeline ``` data/raw/R*/facilities_*.json (one JSON per research agent) │ ▼ scripts/ingest.py │ - dedup (URL match, fuzzy name, 5 km haversine) │ - geocode missing addresses (Nominatim, on-disk cache) │ - load vocab from schema/vocab/*.csv │ - INSERT OR REPLACE into facilities, locations, area_links, │ network_membership, funders, funding_links │ - call populate_regions for spatial linkage │ ▼ scripts/populate_regions.py │ - read every public/overlays/*.geojson │ - INSERT 1 row per polygon into `regions` │ - seed `region_area_links` from per-kind heuristics │ - STRtree point-in-polygon: every facility × every │ region → `facility_regions` containment edges │ ▼ scripts/export_parquet.py - COPY each table TO db/parquet/
.parquet - mirror to public/parquet/ for DuckDB-Wasm HTTP-range reads - emit public/facilities.geojson as a lightweight fallback ``` `scripts/qa.py` runs a bounding-box, enum, foreign-key, and provenance audit and writes `data/raw/validation-report.md`. ## In-browser data access The app first paints from a lightweight `public/facilities.geojson` fallback so something is on the screen in under a second. In parallel it downloads DuckDB-Wasm, opens the parquet files in `public/parquet/` over HTTP range requests (no server needed), and re-runs the query to pick up research areas, networks, funders, and region membership. Once DuckDB is up, every filter change re-issues the SQL without re-downloading any parquet chunks. Parquets loaded into the browser DuckDB: `facilities`, `locations`, `funders`, `funding_links`, `research_areas`, `area_links`, `networks`, `network_membership`, `regions`, `region_area_links`, `facility_regions`, `people`, `facility_personnel`, `person_areas`, `person_area_metrics`, `person_primary_groups`, `publications`, `authorship`, `publication_topics`, `collaborations`, `provenance`, `person_registry`, `person_identity_source`, `registry_collaborations`, `registry_facilities`. Note that the registry parquet in `public/parquet/` is not the whole table. Only the 10,000-row `core` tier is served to the browser, along with the collaboration edges and facility links whose endpoints are both in that tier. The full 152,008-row population lives in `db/parquet/` and is queryable only against a local DuckDB — so any count read off the site's SQL tab is a count within the core tier, not within the field. [The Person Registry](#/docs/person-registry) explains what that selection is and is not. ## Deduplication `scripts/ingest.py` merges records using a three-step check: 1. **Exact URL match.** Two records with the same canonical URL are always merged. 2. **Fuzzy name match.** RapidFuzz `token_set_ratio ≥ 92` across `canonical_name`. 3. **Proximity check.** If (1) or (2) triggered, also require a haversine distance < 5 km between reported HQs before merging. When merging, each field is kept from the record with the higher confidence rating in its `provenance` block (high > medium > low). List-valued fields (locations, research areas, networks, funders) are unioned. ## Tech stack | Layer | Technology | |---|---| | Data storage | DuckDB + Parquet exports | | Spatial linkage | shapely 2.x with STRtree | | Ingest | Python 3.11; `duckdb`, `rapidfuzz`, `geopy`, `shapely` | | In-browser query | DuckDB-Wasm via esm.sh, HTTP range reads | | Map | MapLibre-GL 4.7.1, OpenFreeMap positron tiles | | Front end | Vanilla JavaScript ES modules — no bundler | | Hosting | GitHub Pages — repo root served directly | ## Known gaps and future work - **CCAP 2010 raster land cover** — we have the metadata but the raster files are too large for GitHub without LFS; not yet rendered. - **Per-NERR salt-marsh habitat** — habitat sub-types and elevation exist under each reserve but aren't yet exposed in a per-reserve drill-down UI. - **NPS administrative regions** (the seven regional offices, not the coastal units) — not rendered. - **California wetland potential** — available as a shapefile, not yet on the map. - **Funder coverage** — still being filled in for non-NSF federal facilities. - A handful of South American and Caribbean facilities are missing coordinates and don't appear on the map. ## License and attribution Code is MIT-licensed. Data carries per-source attribution in every overlay popup and in the repository's `LICENSE` file. Upstream spatial archive: COMPASS-DOE/synthesis-networks (MIT). Basemap: [OpenFreeMap](https://openfreemap.org/) positron, © OpenMapTiles, data © OpenStreetMap contributors. ------------------------------------------------------------------------ ## Team, scholars & data URL: https://tyson-swetnam.github.io/cod-kmap/docs/team_scholars_datasets_methods.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/team_scholars_datasets_methods.md # Team, Scholars & Data — methods and provenance This page documents how three of cod-kmap's tabs are built: the **Team** tab (the COD project org chart), the **Scholars** tab (the coastal ocean science research community), and the **Data** tab (curated coastal datasets and their access endpoints). All three were proposed in [the reference documents report](#/docs/reference-documents-report) and built in July 2026. Each is regenerable from committed seed files — no step depends on state that only exists on one machine. --- ## 1. COD project team (the Team tab) ### Where the data comes from The project organisational chart (2026 revision) is transcribed by hand into two CSVs: | File | Contents | |---|---| | `data/seed/cod_wbs.csv` | 52 work-breakdown elements: code, parent, title, display order | | `data/seed/cod_team_members.csv` | 67 rows — one per (person, WBS element, role) | A person who leads several WBS elements gets one row each, which is how the chart's "5.0 / 5.1 / 5.6" pattern survives being flattened into a table. The Team tab recombines them into a single card. The roster covers 40 named people, 16 unfilled positions, and one group-staffed element. `status` distinguishes them: | status | meaning | synced into `people`? | |---|---|---| | `active` | a named individual | yes | | `tbd` / `tbh` | an unfilled position, rendered muted | no | | `collective` | work staffed by a group, not a person — the chart's "NEON Staff" box | no | Unfilled positions are rendered muted rather than hidden, because an empty slot is information about the project rather than missing data. `collective` exists so a staffing pool is not stored in `people` as though it were a human: it was, briefly, and the enrichment scripts would have gone looking for its publications. ### Institution slugs `institution_slug` mirrors the chart's colour legend and is validated against `COD_INSTITUTIONS` in both the build script and `scripts/qa.py`, so a typo cannot silently produce an uncoloured chip: ``` clemson yale unm battelle vcu pnnl unl obfs uga arizona delaware usc uidaho montana-state alabama florida coastal-carolina charleston other-university agency company various ``` ### Two chart labels worth confirming For two people the chart's printed affiliation differs from the one they are otherwise associated with. The seed records the institutional affiliation and notes the chart's text, so the discrepancy is visible rather than silently resolved — worth checking against the award documents: | Person | Recorded here | The 2026 chart prints | |---|---|---| | Rodrigo Vargas | University of Delaware | Arizona | | Christine Angelini | University of Florida | AECOM | Some names were also read off the chart image, so spellings are worth a pass — the chart renders "Maclamore" where Clemson lists Eric McLamore. ### Building it ```bash python scripts/build_cod_team_lake.py ``` This is idempotent: re-running with unchanged seeds produces identical tables and identical parquet. ### Why a DuckLake The team roster is the one table in this repo that is expected to change by revision rather than by re-harvest — people join, positions get filled, the WBS gets reorganised — and "what did the team look like at proposal time?" is a question worth being able to answer. So the build writes into a **DuckLake** catalogue: ``` db/cod_team.ducklake the catalogue db/ducklake_data/ DuckLake-managed parquet ``` Every run lands as a snapshot, so the history is queryable: ```sql ATTACH 'ducklake:db/cod_team.ducklake' AS teamlake; SELECT * FROM teamlake.snapshots(); SELECT * FROM teamlake.cod_team_members AT (VERSION => 1); ``` Both paths are gitignored, for the same reason `db/cod_kmap.duckdb` is: DuckDB's on-disk format is not portable across versions, and the whole thing is regenerable from two CSVs. The **shared** artifact is still plain parquet in `db/parquet/` and `public/parquet/`, which is also what DuckDB-Wasm reads in the browser. Snapshot history is a local convenience, not shared state. Rows are staged and moved with a single `INSERT … SELECT` per table. DuckLake records one snapshot per statement, so inserting row-by-row would bury each run's real change under a hundred single-row snapshots. The extension is optional. Install it without needing `extensions.duckdb.org`: ```bash pip install duckdb-extensions duckdb-extension-ducklake ``` If it is unavailable the script says so and writes plain tables into the main database instead. The parquet output is identical either way, so a fallback run is not a degraded run — you just lose the snapshot log. ### Syncing into `people` Named members are upserted into `people` on `person_id = sha1(lower(name) | orcid | lower(email))[:16]` — the same formula as `scripts/load_facility_personnel.py`, so the same human seeded through either path collapses onto one row. Every enrichable column is written with `COALESCE(excluded.x, people.x)`, so a blank cell in the seed CSV never wipes a value that the enrichment scripts resolved earlier. ### Filling in profiles and metrics The seed CSV ships with Google Scholar IDs for the PI and Co-PIs and blank ORCID / OpenAlex columns for everyone else. **A missing identifier is much better than a wrong one**: attaching the wrong ORCID to a researcher misattributes their entire publication record, and this repo has already had to undo exactly that (see `scripts/wipe_bad_openalex_attributions.py`, which cleaned up after a name-only resolver attached cardiologists to marine laboratories). To fill them in, run the existing enrichment chain against a network that can reach `api.openalex.org` and `pub.orcid.org`: ```bash export OPENALEX_EMAIL=you@example.org # OpenAlex polite pool python scripts/enrich_people_orcid.py # strict 3-rule matcher python scripts/enrich_people_openalex.py # publications + topics python scripts/enrich_people_gscholar.py # Scholar ids via OpenAlex/ORCID python scripts/backfill_publication_topics.py python scripts/compute_person_areas.py # needs publication_topics above python scripts/compute_collaborations.py --export-parquet python scripts/compute_primary_groups.py # MUST precede area_metrics python scripts/compute_area_metrics.py # h-index, citations, composite_z python scripts/init_people_tables.py --export-parquet # publications/authorship/topics python scripts/build_cod_team_lake.py # re-snapshot people.parquet python scripts/build_person_registry.py # unify the three layers on one key python scripts/compute_registry_collaborations.py # cross-cohort co-pub edges python scripts/link_registry_facilities.py # researcher ↔ site, on ROR equality python scripts/rank_person_registry.py # assign core / archive tier python scripts/qa.py ``` The four registry scripts run last, and in that order: the registry needs the three source layers populated before it can unify them, the graph and the facility links need the registry's node ids, and tiering scores collaboration degree so it has to follow the graph. Three things about that order are easy to get wrong, and were wrong in an earlier version of this page: - **`compute_primary_groups.py` must run before `compute_area_metrics.py`**, not after. Two of the metric tables join `facility_primary_groups`, which only the groups script produces. - **`compute_collaborations.py` needs `--export-parquet`**; without the flag it updates the database and writes no parquet, so the co-author counts never reach the browser. - **`init_people_tables.py --export-parquet` is not optional.** No script in the chain exports `publications`, `authorship`, `person_areas` or `publication_topics`, so newly harvested publications would sit in the local database and never reach the site. That command re-exports all seven people-side tables (it is `CREATE TABLE IF NOT EXISTS`, so it will not wipe anything). Then stage the refreshed parquet — it is gitignored but tracked, so a plain `git add` silently skips it: ```bash git add -f db/parquet/*.parquet public/parquet/*.parquet ``` Until that runs, the Team tab shows profile links but no metrics, and says so at the foot of the page. --- ## 2. Coastal ocean science scholars (the Scholars tab) `community_scholars` is a field-wide roster: who defined coastal ocean science, who is publishing most in it right now, and who is coming up. It is deliberately a **separate table from `people`**. `people` is the staff of catalogued facilities; these researchers mostly do not work at one, and mixing a bibliometric cohort into the facility directory would distort every per-facility metric on the Stats tab. **Separate table, but no longer a separate identity space.** Until the registry work, being separate tables also meant being unlinked: a researcher who was both facility staff and a roster scholar was two rows with two keys, and nothing in the schema could say they were one person. `person_registry` now resolves `people`, `cod_team_members` and `community_scholars` into one node set keyed on a persistent identifier, so cross-cohort questions — who on the project team already publishes with whom on the roster — are answerable. Eleven people turn out to hold more than one cohort flag. The source tables keep their own grain and their own columns; the registry adds the shared key. See [The Person Registry](#/docs/person-registry). ### Cohorts | Flag | Meaning | |---|---| | `is_preeminent` | Established, field-defining, highly cited | | `is_most_active` | Among the most prolific in the last five years | | `is_rising` | Early career, rapidly growing impact | A scholar can carry more than one flag. Each has its own rank column, which is what the Scholars tab orders on. **A rank is only populated once the scholar has been measured.** The curated roster ships with all three rank columns NULL, and the tab renders the cohort badge without a number. An earlier version filled them in alphabetically, since there was nothing else to sort on — which rendered as "Pre-eminent #1" for a surname beginning with A and read as a finding rather than an artifact. `scripts/qa.py` enforces the weaker, honest invariant: a rank may not exist without its flag, ranks must be unique, and every *measured* row in a cohort must be ranked. **The curated roster is a candidate pool, not the final cohorts.** A wide pool gives the harvest more to rank and makes it less likely that a genuinely leading researcher is missing entirely. The harvest then pins the cohorts to the sizes in `COHORTS` — 100 pre-eminent, 100 most-active, 50 rising — and `scripts/qa.py` enforces those sizes once measured rows exist. The harvest has now run. The table holds **523 rows**, of which **220 carry harvested metrics**; 303 remain curated-only with null metrics. The 220 is the cohort total: 100 + 100 + 50 with 30 scholars holding both the pre-eminent and most-active designations. Curated-only rows are kept rather than dropped, and the tab labels them "bibliometrics pending" rather than rendering zeros. ### Two ways the table gets populated **Curated (the seed).** Scholars researched by hand across ten sub-fields — physical oceanography, estuarine ecology, coastal geomorphology, sea level, blue carbon, HABs and water quality, ocean observing, coastal hazards and engineering, fisheries and MPAs, and the social dimension — each with an affiliation, sub-field topics, and a one-line rationale. ```bash python scripts/build_community_scholars.py --seed ``` Rows are marked `source = 'websearch-curated'` with metric columns NULL, and the tab labels them "bibliometrics pending" rather than rendering zeros. `confidence` records how well the identity was corroborated; `low` means the identity check did not complete, not that the person is doubtful. **Harvested (measured).** Needs `api.openalex.org`: ```bash export OPENALEX_EMAIL=you@example.org python scripts/build_community_scholars.py --harvest ``` Five stages, each checkpointed under `data/raw/community_scholars/` (gitignored) so `--resume` is cheap and `--stage A|B|C` bounds a run: | Stage | What it does | |---|---| | A | Per topic, group `/works` by author to find who publishes most in it; union across topics | | B | Hydrate authors 50 at a time: ORCID, summary stats, last known institution | | C | For the shortlist only, count coastal-topic works all-time and over five years, and find each author's first publication year | | D | Assign cohorts locally and deterministically | | E | Link to `people`, write the table, refresh parquet | About 1,250 requests, minutes in the polite pool. ### The topic set is the reproducibility anchor Cohorts are defined relative to `data/datasets/coastal_topics.csv` — 18 OpenAlex topics spanning the sub-fields above. Change that file and the cohorts change, which is why it is committed alongside the roster it produced. Topic IDs are resolved once and pasted back in: ```bash python scripts/build_community_scholars.py --resolve-topics ``` The harvest **refuses to run** while any row still holds the `RESOLVE` sentinel, so a published roster is always traceable to explicit topic IDs rather than to whatever a search happened to return that day. ### Identity rules Two rules, both there because this repo has been burned before: 1. **Candidates are OpenAlex author IDs from the start.** No name matching ever happens, at any stage. 2. **A coastal-share gate**: an author needs at least 10 works in the topic set *and* at least 15% of their total output inside it. This is what stops a prolific researcher in an unrelated field from ranking on total h-index after one coastal paper. A scholar is linked to an existing `people` row only on ORCID or `openalex_id` equality — never on name. The same rule governs `person_registry`, which is where that link is now materialised as a shared node id rather than left implicit. ### Curated and harvested rows reconcile A curated scholar matched by ORCID or Google Scholar ID keeps their curation rationale and gains measured metrics. An unmatched curated scholar is **kept**, not dropped: a hand-picked expert should not vanish because a threshold did not like them. --- ## 3. Curated coastal datasets (the Data tab) `data/datasets/coastal_datasets.json` is the single source of truth. The loader deletes and re-inserts both tables on every run, so editing the JSON and re-running is the entire update workflow: ```bash python scripts/load_coastal_datasets.py python scripts/load_coastal_datasets.py --dry-run # validate only python scripts/load_coastal_datasets.py --check-urls # probe endpoints (needs network) ``` ### What's in it 72 datasets with 242 access endpoints, 62 of them exposing a machine-readable service: - the programs named in the Design Flow diagram — MarineGEO, NERRS/CDMO, C-CAP, Coastal Zone Management, Digital Coast, IOOS, NASA coastal products and the Sea Level Change portal, EPA NCCA, the Critical Zone network, Coastal Carbon, coastal LTER, OOI, NEON, CODISS, CDIP - all 11 IOOS regional associations, each with its own ERDDAP - federal archives and APIs — NDBC, CO-OPS, NCEI, CoastWatch, USGS CMHRP and ScienceBase, PO.DAAC, Earthdata/CMR, OBIS, GBIF, BCO-DMO, HydroShare - a few international counterparts (Copernicus Marine, EMODnet) ### Endpoints are the point A dataset without an access endpoint is exactly what this catalogue exists to prevent, and `scripts/qa.py` fails if one appears. Each endpoint carries a type from a fixed vocabulary: ``` erddap thredds opendap ogc-wms ogc-wfs ogc-api rest-api s3 ftp portal doi stac ``` Everything except `portal` is a machine-readable service. The Data tab renders one badge per endpoint — clicking opens it, the ⧉ button copies the URL — and colours `portal` badges muted so the API endpoints read first. Datasets are also categorised, which is what the tab filters on: ``` observing-system monitoring-program data-portal remote-sensing synthesis-network model-output archive mapping-product ``` ### Curation rules - Every dataset needs at least one endpoint and should have a `portal` landing page alongside any machine endpoint. - `network_id` links to `schema/vocab/networks.csv` where a slug exists. Do not invent network slugs to make a link work; the loader nulls an unknown one and warns. - `confidence` is `high` for a verified endpoint on the provider's own site, `medium` where the program is real but the access route is spread across several services, `low` where no public endpoint could be confirmed at all (CODISS is the one such entry — the DHS system it names is not open data). - `program` labels must be consistent, because the tab groups on them. The catalogue was assembled from two independent research passes that named the same programs differently; the labels are canonicalised so real families group together instead of splitting into singletons. - URLs are recorded as the provider publishes them. `--check-urls` probes them but is off by default, since it needs outbound network access that CI does not have. --- ## Verifying all three ```bash python scripts/rebuild_db_from_parquet.py # committed parquet -> local DB python scripts/qa.py # must exit 0 python -m http.server 5173 # then open /#/people, /#/org, /#/data ``` `scripts/qa.py` carries 13 invariants for these tables — exactly one PI, at least three Co-PIs and ten committee members, named members resolving to `people` rows, unfilled positions carrying no person, WBS and parent-code closure, cohort flags agreeing with cohort ranks, ORCID and OpenAlex ID shape, no dataset without an endpoint, endpoint foreign-key closure, and vocabulary membership throughout — plus a column-presence check per table, which catches the schema-versus-loader drift that has already bitten the people tables once. Every block is skipped when its table is empty, so the weekly ingest-only refresh (where these tables have no rows, because their data lives in committed parquet that `ingest.py` never touches) stays green. ------------------------------------------------------------------------ ## Person registry URL: https://tyson-swetnam.github.io/cod-kmap/docs/person_registry.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/person_registry.md # The person registry — one identity space for three cohorts cod-kmap describes people in three different ways. Some are staff at a catalogued facility. Some sit on the COD project organisational chart. Some are members of a field-wide roster of coastal ocean science researchers, most of whom have no connection to a catalogued site at all. Those three descriptions were built independently, at different times, from different sources, and each got its own table. That was workable for as long as the only questions being asked were per-cohort — *who staffs this reserve?*, *who leads this work-breakdown track?*. It broke down as soon as the question spanned cohorts. **Who does the project team already publish with?** is a question about all three at once, and it was unanswerable: the same researcher could appear as three rows with three unrelated keys and nothing connecting them. `person_registry` is the table that fixes that. It is a single identity space: one row per human, keyed on a persistent identifier, carrying a membership flag for each cohort the person belongs to. ## What the table is, and what it is not The registry does **not** replace the three source tables. Those keep their own columns and their own grain — a team member still has a work-breakdown code and a role; a roster scholar still has cohort flags and ranks; facility staff still have a role at a named site. What the registry adds is a common node identifier, so that anything computed over people can be computed **once**, over one node set, rather than three times over three disconnected ones. The co-publication graph is the clearest example. The older `collaborations` table is keyed on facility-staff identifiers, so it is structurally incapable of expressing an edge between a project team member and a roster scholar — not because no such collaborations exist, but because the table has nowhere to put them. `registry_collaborations` is keyed on the registry, so it can. ## What the canonical key means Every row is keyed on `canonical_id`, a string of one of two forms: | Form | When it is used | |---|---| | `orcid:0000-0000-0000-0000` | The person has a known ORCID | | `openalex:A1234567890` | No ORCID is known; an OpenAlex author id is | Of the 152,008 identities in the local database, 86,620 are keyed on an ORCID and 65,388 on an OpenAlex author id. A row must carry at least one of the two — a registry entry with neither could never be re-resolved or de-duplicated on a later run, so the quality gate rejects it. The key is derived from the identifier itself, not from a hash of mutable fields such as name, email, or affiliation. That means it is stable across rebuilds: re-running the pipeline on refreshed source data produces the same `canonical_id` for the same person, so links built on top of the registry survive a refresh. ### Two rows merge only on identifier equality This is the single most important rule in the table, and it is worth stating plainly because it constrains what the registry can and cannot do for you. **Two records merge into one identity only on a shared identifier.** Name similarity never merges anything — not a high fuzzy-match score, not an exact name match, not an exact name match at the same institution. Only ORCID or OpenAlex-author-id equality. The rule is not conservatism for its own sake. This repository has had to undo wrong-person attributions more than once, and the failure mode is severe: attaching one researcher's identifier to another's row misattributes their entire publication record, and every metric computed downstream — h-index, citation count, coastal output, collaboration degree, cohort rank — is then wrong in a way that looks entirely plausible on the page. A missing identifier costs a blank field. A wrong identifier costs the credibility of every number next to it. The consequence to keep in mind when reading the site: two rows with the same display name may be the same human. If neither carries a shared persistent identifier, the registry will not assert that they are, and the interface will not merge them. Deliberate under-merging is the intended behaviour. Every identifier the registry holds is traceable to the rule that put it there. A companion table, `person_identity_source`, carries one row per identifier assertion — 152,653 of them — naming the resolution method, the evidence, the source URL, and a confidence rating. A wrong identifier can therefore be traced back to the rule that produced it rather than being silently overwritten. Refusals are handled asymmetrically, and it is worth knowing which is which. Identifier *conflicts* are written into `person_identity_source` as first-class rows. Identifier *resolution refusals* — a name that could not be tied to one author safely — are reported by the resolver and left as a null column; they are not rows in this table. So an absent ORCID on a registry row does not carry its own explanation, and "why does this person have no ORCID?" is answered from the resolver's run output rather than from the database. ## Two tiers, and what the site actually shows The application queries parquet files directly in the browser; there is no query server. Everything the interface can see has to be downloaded to the reader's machine first. A 152,008-row researcher table with full bibliometrics is not something to hand to a browser on page load. So the registry is tiered: | Tier | Rows | Where it lives | |---|---:|---| | `core` | 10,000 | Shipped to the site; queryable in the browser | | `archive` | 142,008 | Local catalogue only | **The site shows the core tier and only the core tier.** This is the most important thing to understand about any figure you read off a registry view. A count of researchers in a country, a topic, or a facility is a count within those 10,000 rows, not within the field. Nor is `core` a random sample, so it cannot be treated as one and scaled up. Rows were ranked on percentile ranks of coastal output volume, h-index, recent citation impact, and collaboration degree, and the top 10,000 were taken. Percentile ranks rather than raw values, because ranking on raw counts put prolific generalists with large non-coastal output above genuine coastal specialists. The two tiers therefore differ systematically, exactly as intended: | Measure | `core` | `archive` | |---|---:|---:| | Rows | 10,000 | 142,008 | | Mean h-index | 39.5 | 10.6 | | Mean coastal output volume | 94.9 | 13.7 | | Countries represented | 118 | 201 | Read that table as a warning about selection, not as a finding about researchers. The core tier is the high-visibility end of a citation-weighted ranking. Early-career researchers, researchers publishing in languages and venues OpenAlex indexes less completely, and researchers at institutions with smaller publication throughput are systematically more likely to be in the archive tier. The 84 countries that appear in the archive tier but not the core tier are not absent from coastal science. One exception is deliberate: all 618 identities that were in the registry before the field-wide harvest — the facility staff, the project team, and the curated roster — are pinned into `core` regardless of score. A metric threshold must never drop the roster the site exists to show. ### Cohort membership in what ships Three boolean columns record which cohorts a person belongs to, and a person can carry more than one flag at once. In the 10,000 shipped rows: | Flag | Rows in `core` | |---|---:| | `is_scholar` — field-wide roster | 9,824 | | `is_site_personnel` — staffs a catalogued facility | 173 | | `is_team` — on the COD organisational chart | 14 | Eleven people in the full registry carry more than one flag — several are simultaneously facility staff and roster scholars, a fact none of the three source tables could represent on its own. That overlap is the reason the registry exists. Identifier coverage within the shipped tier: 9,309 rows carry an ORCID, 9,996 an OpenAlex author id, 9,447 a ROR-identified affiliation, 219 a homepage URL, and 12 a Google Scholar id. ## Links from researchers to catalogued sites Facilities previously had no persistent organisation identifier, so a researcher's affiliation — which OpenAlex reports with a ROR id — had nothing to join against. A `ror` column was added to `facilities`, and `registry_facilities` holds the resulting researcher-to-site links, joined on ROR equality alone. No name matching is involved here either. Coverage is partial and should be read as a floor: - 69 facilities have a resolved ROR, out of the 210 records that are research organisations. The remaining organisations were not attempted rather than found to have none — the identifier lookup ran out of API budget, and re-running it extends coverage with no other change. - The 3,309 protected-area records — state parks, refuges, wilderness units, aquatic preserves — are excluded by design and hold no ROR. A protected area is a place, not an organisation; it will never hold a ROR, and its null column is correct. - 1,467 researcher-to-site links exist across 39 sites in the local catalogue. Only 263 of them, across 26 sites, reach the site, because the other 1,204 point at archive-tier researchers who are not shipped. An absent link therefore means one of three different things — the facility has no ROR yet, the researcher's affiliation is elsewhere, or the researcher is in the archive tier — and the interface cannot distinguish them for you. ## Data caveats These are properties of the data, not defects awaiting a fix. Anything you build on the registry needs to account for them. ### Coastal output volume is an upper bound, not a paper count `coastal_works_count` is the sum of a researcher's work counts across the coastal topic set. OpenAlex assigns a work to *every* topic it carries, so a paper spanning three coastal topics is counted three times in that sum. The number is the best available signal of **coastal output volume**, and it is useful for ranking, but it is not a count of distinct papers and must not be labelled as one. The build recorded the scale of that double-counting directly: before the value was clamped to the researcher's own total work count, it exceeded that total for 10,571 rows, reaching roughly three times it at the extreme. The shipped column is clamped, so the inflation is no longer visible in the data — but it is still in the measure. `coastal_share`, derived from it, inherits the same inflation and should be read the same way. ### The co-publication graph covers a subset of the nodes `registry_collaborations` holds 5,300 co-publication edges. Building the graph over all 152,008 identities would require on the order of 152,000 publication queries against an external API, so it was computed over the 618 identities that were in the registry before the field-wide harvest — the COD-relevant subgraph. The practical consequence: **510 nodes carry at least one edge.** The other 9,490 of the 10,000 shipped rows have none. For nearly every core-tier researcher, an empty collaboration list means *the graph has not been computed for this person*, not *this person has no collaborators*. Those two statements are entirely different and must never be conflated — an interface that renders the first as the second is reporting an artifact of compute budget as a finding about a researcher's career. The same asymmetry propagates into tier scoring: collaboration degree contributes to a row's score only for nodes that were in the registry before the harvest, and is silently zero for the rest. ### Two ORCID conflicts are logged, not resolved Two identities exist where an external source reports a different ORCID than the registry holds. Neither has been applied. Both sit in `person_identity_source` with `field = 'orcid-conflict'`, awaiting human curation. Overwriting an identifier on the strength of one source disagreeing with another is exactly the operation that causes wrong-person attribution. ### Affiliation strings are whatever the source last attached `affiliation` is the display string from the OpenAlex author record, reproduced as-is. It is sometimes wrong, or a department name where an institution is expected, even when the bibliometrics on the same row are correct. `affiliation_ror`, where present, is the more reliable field and is what the facility join uses. Treat the free-text string as a label, not as evidence. ### Google Scholar ids are effectively absent Twelve of 152,008 rows carry one. OpenAlex does not populate a Scholar identifier for most authors, and there is no other free deterministic source, so this is a property of the upstream data rather than a gap in the pipeline. Filling the column would require a different source or a hand-curation pass. ### Career-stage fields are unreliable at the tails `first_pub_year` comes from the earliest work on the author record, and a small number of records carry implausible values — years in the 1800s arising from a mis-dated indexed work. Any analysis keyed on career stage should filter these rather than assume the column is clean. `two_yr_mean_citedness` measures recent citation impact, which is a proxy for recent activity and not a measure of recent output volume. ## Where this is defined The table definitions, including the reasoning above in comment form, are in [`schema/schema.sql`](https://github.com/tyson-swetnam/cod-kmap/blob/main/schema/schema.sql). The registry and its companion tables are built by `scripts/build_person_registry.py`, tiered by `scripts/rank_person_registry.py`, given a co-publication graph by `scripts/compute_registry_collaborations.py`, and linked to facilities by `scripts/link_registry_facilities.py`. `scripts/qa.py` enforces the invariants described here — identifier uniqueness, at least one persistent identifier per row, provenance for every assertion — and is the gate a rebuild has to pass. The full 152,008-row population is exported to `db/parquet/`; only the core tier and the links whose endpoints ship are mirrored to `public/parquet/`, which is what the browser reads. ------------------------------------------------------------------------ ## Validation report URL: https://tyson-swetnam.github.io/cod-kmap/docs/VALIDATION_REPORT.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/VALIDATION_REPORT.md # Registry validation report Run `val-20260729T035944Z` · 2026-07-29 Figures in this report were produced interactively and then reproduced by `scripts/validate_registry.py`, which was written from that work and is the reusable entry point going forward. Where a figure comes from a superseded run or a different scope, that is stated at the figure. Validates all 10,095 core-tier researchers in `person_registry` against their persistent identifiers, resolves their ROR affiliations, harvests coastal-topic publications, and matches the resulting co-authors back to the registry through an OWL identity layer. **QA status: passing** (`python scripts/qa.py`), with three new tables and their check functions in place. --- ## 1. Identity validation — 40,380 verdicts Four checks per researcher, long form keyed on `(canonical_id, check_id, run_id)` so a re-validation appends rather than overwrites and two runs can be diffed. | check | pass | fail | not_applicable | unresolved | |---|---:|---:|---:|---:| | `openalex-author-resolves` | 9,996 | 0 | 99 | 0 | | `orcid-resolves` | 9,298 | 2 | 786 | 9 | | `ror-resolves` | 9,443 | 4 | 648 | 0 | | `name-agreement` | 9,995 | 0 | 99 | 1 | Every row carries a `source_url` and a confidence in {high, medium, low}. Confidence: 34,996 high, 4,980 medium, 404 low. **The identity layer is cleaner than expected.** All 9,996 OpenAlex author ids resolved on the first pass with zero stale or merged records, and after Unicode-aware name folding there are zero full name disagreements (9,869 exact, 126 family-name-only, 1 partial). This is a well-maintained registry. ### The 2 ORCID conflicts — needs a human decision | researcher | stored | OpenAlex | |---|---|---| | Kate Moran | `0000-0002-0353-3385` | `0000-0001-5023-0259` | | Julia Azanza Ricardo | `0009-0004-7818-8696` | `0000-0002-9454-9226` | Both look like duplicate ORCID registrations for the same person rather than false matches. **Not auto-corrected** — picking a winner between two ORCIDs is a curation decision, and the losing id may be the one cited in published work. Both values are recorded in `person_validation.mismatch_detail`. ### The largest defect class: 114 inactive or withdrawn RORs Every one of the **2,537** distinct ROR ids stored on core registry rows resolves — zero invalid. (The lookup cache holds 5,270 resolved RORs: those 2,537 plus 2,733 more that came from OpenAlex `last_known_institutions` during the comparison. Only the 2,537 are the registry's own ids; the 5,270 is the total resolution workload.) But 114 researcher rows cite an organisation whose ROR registry status is no longer active (109 inactive, 5 withdrawn). Those 114 rows point at just **28 distinct organisations**, so a remapping pass is small work with broad effect: | ROR | organisation | status | rows | |---|---|---|---:| | `026nh4520` | CSIRO Oceans and Atmosphere | inactive | 38 | | `01bpa4157` | Institut Català de Ciències del Clima | inactive | 23 | | `04hxcaz34` | National Institute of Water and Atmospheric Research | inactive | 6 | | `028cdc266` | ARC Centre of Excellence for Coral Reef Studies | inactive | 5 | These are historically correct and resolve fine; they should be remapped to successor records. **Recommended action:** a follow-up pass that reads `relationships` → `successor` from the ROR API and proposes remappings for review. ### One clear data error `Yan Jin` is stored with ROR `009hj8759` — **Grady Memorial Hospital**, a hospital — where OpenAlex reports University of Georgia. This is one of 4 `ror_stale` rows and the only one that is unambiguously wrong rather than merely out of date. --- ## 2. Co-author graph Harvest bounded to a curated coastal topic set (§3) and capped at 200 works per researcher — a scope decision forced by measurement, not preference: the OpenAlex key carries a hard quota of 10,000 requests/day and throughput is server-capped at ~62 works/s regardless of concurrency, so parallel workers split bookkeeping rather than wall time. | | | |---|---:| | co-author pair rows harvested | 3,446,537 | | researchers covered | 3,000 of 9,996 (30.0%) | | distinct co-authors surfaced | 335,070 | | matched into `person_registry` | 81,276 | | registry-to-registry edges | 626,200 | | out-of-registry candidates | 253,794 | Every one of the 626,200 edges names the OpenAlex Work that proves it (`exemplar_work_id`, NOT NULL by schema constraint) and the identifier rule that matched it (`match_method`). **`match_method` admits only identifier equality.** The schema CHECK constraint permits `orcid-equality`, `openalex-author-id-equality` and `sameas-closure` — there is no `name-similarity` member and one must never be added. Name matching is the false-match mode this repo has cleaned up repeatedly. ### The archive tier is doing real work Of the 81,276 co-authors that resolve into the registry, only **9,805 are core tier** — the other **71,471 are archive-tier identities the project already holds**. Matching against the core tier alone would have mislabelled every one of those 71,471 as a new discovery. This is the single most important reason `coauthor_candidates` must be built against the full 152,103-row registry and not the 10,095 rows the browser sees. 462 of the matches came from ORCID equality where OpenAlex-id equality failed — identities a single-key join would have missed. ### Expansion candidates 253,794 co-authors are in neither tier. Ranked by breadth of COD collaboration, publication volume, ORCID presence, and whether their affiliation ROR matches a catalogued facility: | suggested confidence | count | |---|---:| | high | 3,644 | | medium | 39,711 | | low | 210,439 | 3,194 are already ROR-affiliated with a catalogued COD facility. Every row carries `decision='pending'`, a `seen_with_canonical_id` naming the registry member it was seen with, and a `seen_on_work_id` proving the co-authorship. **Nothing here is a personnel record** until a curator promotes it. ### What ships to the browser `public/parquet` gets the core-to-core edge subset (213,021 rows, 10.3 MB) and high+medium candidates (43,355 rows, 3.2 MB). The full 626,200-row edge list stays in `db/parquet`: the browser reads `public/parquet` over HTTP and its `person_registry` has only 10,095 rows, so an edge touching an archive-tier person could not be rendered at all. --- ## 3. What the OWL layer actually contributed The honest accounting, because it is easy to overstate this. `cod.owl` declares `cod:orcidId`, `cod:openAlexAuthorId`, `cod:rorId`, `cod:canonicalId`, `cod:openAlexWorkId` and `cod:doi` as `owl:InverseFunctionalProperty`, so an OWL-RL reasoner derives `owl:sameAs` between any two nodes sharing one. Running that closure over the **core-tier-only** graph — 10,095 registry persons plus the 9,015 co-author nodes that a core-only match produced from an earlier, smaller harvest slice (56,188 base triples → 225,217 after closure, 20.9s) — produced 9,015 person↔co-author identity bridges, in exact agreement with the deterministic join over that same slice. These 9,015 figures belong to that closure run and are **not** the core/archive split reported in §2, which was measured separately over the full 335,070-co-author population. **That agreement is not independent validation.** Both methods key on the same identifier equality; the reasoner is a second encoding of the same rule, so agreement is guaranteed by construction and says nothing about whether the linkage is *correct*. **Where the reasoner earns its place: 176 split identities.** These are cases where a co-author shares an ORCID with a registry person but carries a **different OpenAlex author id** — duplicate OpenAlex author records for one human. To be precise about credit: the 176 were *enumerated* by a plain SQL ORCID-equality join, not discovered by the reasoner. What the closure contributes is *resolution* — unifying the two different OpenAlex author ids into one identity, which an OpenAlex-id equality join cannot do (it resolves **0 of 176**, since by construction the two ids differ). Either mechanism can find the pairs; only transitive `sameAs` over an inverse-functional ORCID makes them one person in the graph. This affects 163 distinct registry people, including 2 COD site personnel, and is recorded in `split_identity_findings.parquet`. That is a real, reproducible finding a relational join alone would have missed, and it is the specific justification for keeping the RDF layer. ### SHACL conformance 54 shapes over 12 node shapes, run standalone by `pyshacl` with no reasoner and no network. Against the worked example graph they catch all 11 planted violations with zero false positives — independently reproduced here. Against 400 real registry persons the shapes initially reported **434 violations**, both of them defects in my serialization rather than in the data: a missing `cod:retrievedAt` (a `source_url` without a retrieval date is not reproducible provenance) and a lower-cased ORCID where the shape requires bare 16-digit form. After fixing both: **0 violations, conforms=True**. The shapes did their job. --- ## 4. Topic set — how the harvest was bounded 66 OpenAlex topics, derived empirically from a 303-researcher stratified sample rather than by keyword-matching the project's area labels. Label matching was tested and rejected: it pulled in 5 remote-sensing topics (Soil Moisture and Remote Sensing, 358,880 works) and 3 wildlife-road topics as noise, while returning zero matches for estuaries, wetlands, salt marshes, mangroves or sea level — all of which are project research areas. (Oceanography, fisheries and coral reefs *were* matched.) The derived set closes that recall gap: estuaries, mangroves and salt marshes are covered by `T10779 Coastal wetland ecosystem dynamics`, seagrass by `T10643 Marine and coastal plant biology`. Topics were ranked by **prevalence** — the number of distinct COD researchers publishing in them — not raw work volume, which is what stops one prolific author's niche from entering the set. Two project concepts remain genuinely uncovered and are reported rather than filled with substitutes: - **seabirds** — no OpenAlex topic exists at that granularity - **Great Lakes** — the nearest topic is African Great Lakes limnology, not Laurentian 60 of 66 topics crosswalk to a project `area_id`; the other 6 carry `area_id='NONE'` because no project area was defensible. The confidence distribution (31 high, 28 medium, 7 low) covers all 66 rows including those 6, not the 60 mapped ones. --- ## 5. Coverage limits — read before citing these numbers 1. **The harvest is partial.** 3,000 of 9,996 researchers (30.0%, batches 0-59 of 200) have harvested co-author edges. Batches were ordered most-prolific-first, so the covered fraction holds a disproportionate share of total output, but the co-author graph is **not** complete and edge counts for uncovered researchers are absent, not zero. 2. **Per-author cap of 200 works.** Researchers above that keep their 200 most-cited coastal works. The most prolific researchers — who have the most co-authors — are exactly the ones truncated. 3. **Batch-shared page budget.** Authors are batched 50 per cursor, so a batch's page budget is shared: an author far more prolific than their batch peers can be truncated below 200. 4. **Facility linkage is thin.** Only 263 of 10,095 researchers (2.6%) resolve to a catalogued COD facility by ROR equality. The earlier 413 figure came from an intermediate in-memory column and is not reproducible from the shipped tables; it is withdrawn. 5. **95 codp: rows carry no public identifier** by construction. All their public-identifier checks are `not_applicable`, sourced to the repo's own curated record. They are not defects. 6. **4 people have an ORCID but no OpenAlex id** (Megan Medina, Jill Carr, Josh F.W. Cook, Sheila Lischwe), so their identity could not be cross-checked — rated low confidence, verdict `unresolved`. --- ## 6. Recommended curation actions 1. Resolve the 2 ORCID conflicts (Kate Moran, Julia Azanza Ricardo) by hand. 2. Correct Yan Jin's affiliation — Grady Memorial Hospital is certainly wrong. 3. Remap the 114 inactive/withdrawn RORs to successor records via the ROR API's `relationships` field. 4. Review the 163 split-identity cases; where confirmed, record the duplicate OpenAlex id as an alias rather than a separate identity. 5. Triage the 3,644 high-confidence expansion candidates, starting with the 3,194 already ROR-affiliated with a catalogued facility. 6. Finish the harvest for the remaining researchers once the API quota resets. A parallel run reached batch 85 of 200 (2,793 researchers, 1,945,450 pair rows) in a separate workspace; those batches are **not** merged into the committed tables and would need re-harvesting or transferring. The API key was at 1,640 requests remaining when that run stopped. ## 7. Operational notes for whoever runs this next - **`api.ror.org` was not on the network allowlist** and had to be granted mid-run; the first attempt burned ~66 minutes in retry backoff before the proxy 403 was diagnosed. - **The ROR API now serves v2 schema.** `name` and `country.country_code` are gone: display name is the `names[]` entry typed `ror_display`, country is `locations[].geonames_details.country_code`. Parsing the v1 shape yields silently **empty** names and countries with HTTP 200 — a failure that looks like success. - **OpenAlex throughput does not improve with concurrency.** 1, 4 and 8 threads all measured 60–87 works/s. The key is quota-limited (10,000 requests/day, one credit per request), so budget in requests, not seconds. Batch authors 50 per cursor: same per-work cost, one cursor instead of 50. --- ## 8. Figure provenance — checkable, not asserted Successive drafts of this report claimed broader verification than had been performed. This section is written so a reader never has to take a count on trust. **Run the check yourself:** ``` python scripts/verify_report_figures.py ``` It re-executes the SQL behind every derivable figure and exits non-zero if any disagrees. `db/derived/report_audit_full.json` holds one entry per figure with the reported value, the derived value, and the **exact SQL** that produces it from files tracked in this repository. **49 figures re-derived, zero mismatches**, verified against a clean git worktree at HEAD rather than a working copy. That covers every count in §1 and §2, including the ones earlier drafts skipped: the `not_applicable` columns (99 / 786 / 648), the `fail` and `unresolved` counts (2 / 4 / 9 / 1), the per-organisation ROR row counts (38 CSIRO / 23 ICCC / 6 NIWA), the 28 distinct inactive organisations, the 462 ORCID-only matches, and the 9,805 / 71,471 tier split. Two inputs were **committed specifically so these figures could be re-derived**, having previously existed only outside the repository: - `db/parquet/ror_resolution_cache.parquet` — 5,270 resolved RORs with registry status. No tracked table carried ROR status (`person_validation.evidence` records the verdict only), so the largest defect class — 114 inactive/withdrawn rows over 28 organisations — was uncheckable. - `db/parquet/coauthor_pairs_raw.parquet` — the 3,446,537 harvested pair rows, consolidated from 48 shards under an untracked path. Harvest coverage, the tier split and the ORCID-only match count all depend on it. **12 figures cannot be re-derived from anything in the repository** and are labelled at the point of use rather than counted as verified: the delegated parallel run's totals (1,945,450 pair rows, 2,793 researchers, batch 85 of 200), the live quota reading at that run's exit (1,640 of 10,000), the OWL closure's in-session triple counts (56,188 → 225,217, and 9,015 bridges over the core-only slice), the 434 initial SHACL violations, the per-topic OpenAlex `works_count` (358,880), the measured throughput ceiling (~62 works/s), and the 521 either-side ROR match computed while withdrawing the figure below. These are run-log and API-header observations; re-running the pipeline is the only way to confirm them. **One figure was withdrawn.** An earlier draft reported 413 researchers resolving to a catalogued COD facility. It came from an intermediate in-memory column and is not reproducible from any table: stored-ROR equality gives 263, OpenAlex-returned-ROR equality gives 263, either-side gives 521. The report states 263, matching the committed `registry_facilities` table. Corrections made while building this section, each of which had already reached a committed artifact: - **462 was hardcoded**, assigned from remembered stdout by a cell that counted it among "re-derived" figures. It now comes from a query. - **The tier split was computed against a copy of the gitignored local DuckDB** while being reported as parquet-derived. - **38 and 23 were declared un-derivable** when they are plain row counts of a shipped table. The remaining numbers in this report are dates, section numbers, tuning parameters (per-page 200, batch size 50), ontology term counts, and prose quantities. ------------------------------------------------------------------------ ## References URL: https://tyson-swetnam.github.io/cod-kmap/docs/REFERENCES.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/REFERENCES.md # REFERENCES Citations cataloged in the Coastal Observatory Design Zotero group library — public, browsable, downloadable as BibTeX or RIS at: > https://www.zotero.org/groups/5711743/coastal_observatory_design_nsf/library Zotero group ID: **5711743**. This snapshot captures **98 items** (excluding 74 attachment shells) pulled via the Zotero Public API on 2026-04-26. To regenerate this file from a future snapshot of the group, run: ```sh # 172 items in the group; pulled in pages of 100 (API max). curl -s "https://api.zotero.org/groups/5711743/items?limit=100&start=0&format=json" > /tmp/items_0.json curl -s "https://api.zotero.org/groups/5711743/items?limit=100&start=100&format=json" > /tmp/items_100.json # Or BibTeX directly (auto-handles pagination via Link header): curl -s "https://api.zotero.org/groups/5711743/items?format=bibtex&itemType=-attachment&limit=100" > references.bib ``` A flat BibTeX export of every non-attachment item lives at [`docs/references.bib`](https://tyson-swetnam.github.io/cod-kmap/docs/references.bib) — drop into any LaTeX template with `\bibliography{docs/references}`. ## Journal articles (58) - Thibault, Robert T.; Amaral, Olavo B.; Argolo, Felipe; Bandrowski, Anita E.; Alexandra R, Davidson; Drude, Natascha I., (Oct 19, 2023), **Open Science 2.0: Towards a truly collaborative research ecosystem**, _PLOS Biology 21(10), e3002362_. [doi:10.1371/journal.pbio.3002362](https://doi.org/10.1371/journal.pbio.3002362) - Hopper, Thomas; Meixler, Marcia S., (Oct 12, 2016), **Modeling Coastal Vulnerability through Space and Time**, _PLOS ONE 11(10), e0163495_. [doi:10.1371/journal.pone.0163495](https://doi.org/10.1371/journal.pone.0163495) - Rafiq, Kasim; Beery, Sara; Palmer, Meredith S.; Harchaoui, Zaid; Abrahms, Briana, (2025), **Generative AI as a tool to accelerate the field of ecology**, _Nature Ecology & Evolution, 1-8_. [doi:10.1038/s41559-024-02623-1](https://doi.org/10.1038/s41559-024-02623-1) - National Academies of Sciences; Medicine, (2024), **Tipping Points, Cascading Impacts, and Interacting Risks in the Earth System: Proceedings of a Workshop**. [link](https://nap.nationalacademies.org/catalog/26925/tipping-points-cascading-impacts-and-interacting-risks-in-the-earth-system) - Herndon, Elizabeth; Krauss, Kenneth; Chen, Xingyuan; Steinmuller, Havalend E; Benscoter, Brian, (2024), **Convened by U.S. Department of Energy**. - Herndon, Elizabeth; Krauss, Kenneth; Chen, Xingyuan; Steinmuller, Havalend E; Benscoter, Brian, (2024), **Convened by U.S. Department of Energy**. - Liu, Licheng; Zhou, Wang; Guan, Kaiyu; Peng, Bin; Xu, Shaoming; Tang, Jinyun; Zhu, Qing; Till, Jessica; et al., (2024), **Knowledge-guided machine learning can improve carbon cycle quantification in agroecosystems**, _Nature Communications 15(1), 357_. [doi:10.1038/s41467-023-43860-5](https://doi.org/10.1038/s41467-023-43860-5) - Conroy, Gemma, (2024), **Do AI models produce more original ideas than researchers?**, _Nature_. [doi:10.1038/d41586-024-03070-5](https://doi.org/10.1038/d41586-024-03070-5) - Myers-Pigg; Moanga, Diana; Bond-Lamberty, Ben; Ward, Nick; Megonigal, James Patrick; White, Elliott; Bailey, Vanessa; Kirwan, Matthew, (2024), **Advancing the understanding of coastal disturbances with a network-of-networks approach**, _Ecosphere In Revision_. - Chen, Yaping; Kirwan, Matthew L., (2024), **Rapid greening in mangroves**, _Nature Ecology & Evolution 8(2), 186-187_. [doi:10.1038/s41559-023-02247-x](https://doi.org/10.1038/s41559-023-02247-x) - Crimmins, Allison R.; Avery, Christopher W.; Easterling, David R.; Kunkel, Kenneth E.; Stewart, Brooke C.; Maycock, Thomas K., (2023), **Fifth national climate assessment**. [link](https://repository.library.noaa.gov/view/noaa/61592) - Remington, Thomas F.; Chou, Pallas; Topa, Ben, (2023), **Experiential learning through STEM: Recent initiatives in the United States**, _International Journal of Training and Development 27(3-4), 327-359_. [doi:10.1111/ijtd.12302](https://doi.org/10.1111/ijtd.12302) - Dierssen, H. M.; Gierach, M.; Guild, L. S.; Mannino, A.; Salisbury, J.; Schollaert Uz, S.; Scott, J.; Townsend, P. A.; et al., (2023), **Synergies Between NASA's Hyperspectral Aquatic Missions PACE, GLIMR, and SBG: Opportunities for New Science and Applications**, _Journal of Geophysical Research: Biogeosciences 128(10), e2023JG007574_. [doi:10.1029/2023JG007574](https://doi.org/10.1029/2023JG007574) - Feagin, Rusty A.; Innocenti, Rachel A.; Bond, Hailey; Wengrove, Meagan; Huff, Thomas P.; Lomonaco, Pedro; Tsai, Benjamin; Puleo, Jack; et al., (2023), **Does vegetation accelerate coastal dune erosion during extreme events?**, _Science Advances 9(24), eadg7135_. [doi:10.1126/sciadv.adg7135](https://doi.org/10.1126/sciadv.adg7135) - Vahsen, M. L.; Blum, M. J.; Megonigal, J. P.; Emrich, S. J.; Holmquist, J. R.; Stiller, B.; Todd-Brown, K. E. O.; McLachlan, J. S., (2023), **Rapid plant trait evolution can alter coastal wetland resilience to sea level rise**, _Science 379(6630), 393-398_. [doi:10.1126/science.abq0595](https://doi.org/10.1126/science.abq0595) - Valentine, Kendall; Herbert, Ellen R.; Walters, David C.; Chen, Yaping; Smith, Alexander J.; Kirwan, Matthew L., (2023), **Climate-driven tradeoffs between landscape connectivity and the maintenance of the coastal carbon sink**, _Nature Communications 14(1), 1137_. [doi:10.1038/s41467-023-36803-7](https://doi.org/10.1038/s41467-023-36803-7) - Kirwan, Matthew L.; Megonigal, J. Patrick; Noyce, Genevieve L.; Smith, Alexander J., (2023), **Geomorphic and ecological constraints on the coastal carbon sink**, _Nature Reviews Earth & Environment 4(6), 393-406_. [doi:10.1038/s43017-023-00429-6](https://doi.org/10.1038/s43017-023-00429-6) - Saintilan, Neil; Horton, Benjamin; Törnqvist, Torbjörn E.; Ashe, Erica L.; Khan, Nicole S.; Schuerch, Mark; Perry, Chris; Kopp, Robert E.; et al., (2023), **Widespread retreat of coastal habitat is likely at warming levels above 1.5 °C**, _Nature 621(7977), 112-119_. [doi:10.1038/s41586-023-06448-z](https://doi.org/10.1038/s41586-023-06448-z) - Buscombe, Daniel; Wernette, Phillipe; Fitzpatrick, Sharon; Favela, Jaycee; Goldstein, Evan B.; Enwright, Nicholas M., (2023), **A 1.2 Billion Pixel Human-Labeled Dataset for Data-Driven Classification of Coastal Environments**, _Scientific Data 10(1), 46_. [doi:10.1038/s41597-023-01929-2](https://doi.org/10.1038/s41597-023-01929-2) - Aoki, Lillian R.; Brisbin, Margaret Mars; Hounshell, Alexandria G.; Kincaid, Dustin W.; Larson, Erin I.; Sansom, Brandon J.; Shogren, Arial J.; Smith, Rachel S.; et al., (2022), **Preparing aquatic research for an extreme future: Call for improved definitions and responsive, multidisciplinary approaches**, _BioScience 72(6), 508–520_. [link](https://academic.oup.com/bioscience/article-abstract/72/6/508/6573842) - Patrick, Christopher J.; Hensel, Enie; Kominoski, John S.; Stauffer, Beth A.; McDowell, William H., (2022), **Extreme event ecology needs proactive funding**, _Frontiers in Ecology and the Environment 20(9)_. [link](https://par.nsf.gov/servlets/purl/10385502) - Wicquart, Jérémy; Gudka, Mishal; Obura, David; Logan, Murray; Staub, Francis; Souter, David; Planes, Serge, (2022), **A workflow to integrate ecological monitoring data from different sources**, _Ecological Informatics 68, 101543_. [doi:10.1016/j.ecoinf.2021.101543](https://doi.org/10.1016/j.ecoinf.2021.101543) - Biden Jr, Joseph R., (2021), **Executive Order 13990: Protecting Public Health and the Environment and Restoring Science To Tackle the Climate Crisis**. [link](https://digitalcommons.law.buffalo.edu/cgi/viewcontent.cgi?article=1004&context=reshaping_ej_law_and_social_policy) - National Academies of Sciences; Medicine, (2021), **Next generation earth systems science at the National Science Foundation**. [link](https://nap.nationalacademies.org/catalog/26042/next-generation-earth-systems-science-at-the-national-science-foundation) - Masson-Delmotte, V. P.; Zhai, Panmao; Pirani, S. L.; Connors, C.; Péan, S.; Berger, N.; Caud, Y.; Chen, L.; et al., (2021), **Ipcc, 2021: Summary for policymakers. in: Climate change 2021: The physical science basis. contribution of working group i to the sixth assessment report of the intergovernmental panel on climate change**, _Cambridge University Press, Cambridge, United Kingdom and New York, NY, USA_. [link](http://researchspace.csir.co.za/dspace/handle/10204/12710) - McGill, Bonnie M.; Foster, Madison J.; Pruitt, Abagael N.; Thomas, Samantha Gabrielle; Arsenault, Emily R.; Hanschu, Janaye; Wahwahsuck, Kynser; Cortez, Evan; et al., (2021), **You are welcome here: A practical guide to diversity, equity, and inclusion for undergraduates embarking on an ecological research experience**, _Ecology and Evolution 11(8), 3636-3645_. [doi:10.1002/ece3.7321](https://doi.org/10.1002/ece3.7321) - Malvarez, Gonzalo; Ferreira, Oscar; Navas, Fatima; Cooper, J. A. G.; Gracia-Prieto, F. J.; Talavera, L., (2021), **Storm impacts on a coupled human-natural coastal system: Resilience of developed coasts**, _Science of The Total Environment 768, 144987_. [link](https://www.sciencedirect.com/science/article/pii/S004896972100053X) - Salguero-Gómez, Roberto; Jackson, John; Gascoigne, Samuel J. L., (2021), **Four key challenges in the open-data revolution**, _Journal of Animal Ecology 90(9), 2000-2004_. [doi:10.1111/1365-2656.13567](https://doi.org/10.1111/1365-2656.13567) - Nagy, R. Chelsea; Balch, Jennifer K.; Bissell, Erin K.; Cattau, Megan E.; Glenn, Nancy F.; Halpern, Benjamin S.; Ilangakoon, Nayani; Johnson, Brian; et al., (2021), **Harnessing the NEON data revolution to advance open environmental science with a diverse and data-capable community**, _Ecosphere 12(12), e03833_. [doi:10.1002/ecs2.3833](https://doi.org/10.1002/ecs2.3833) - Pickett, Steward TA; Cadenasso, Mary L.; Baker, Matthew E.; Band, Lawrence E.; Boone, Christopher G.; Buckley, Geoffrey L.; Groffman, Peter M.; Grove, J. Morgan; et al., (2020), **Theoretical perspectives of the baltimore ecosystem study: Conceptual evolution in a social–ecological research project**, _BioScience 70(4), 297–314_. [link](https://academic.oup.com/bioscience/article-abstract/70/4/297/5736085) - Bateman, Ian J.; Mace, Georgina M., (2020), **The natural capital framework for sustainably efficient and equitable decision making**, _Nature Sustainability 3(10), 776-783_. [doi:10.1038/s41893-020-0552-3](https://doi.org/10.1038/s41893-020-0552-3) - Amato, Federico; Guignard, Fabian; Robert, Sylvain; Kanevski, Mikhail, (2020), **A novel framework for spatio-temporal prediction of environmental data using deep learning**, _Scientific Reports 10(1), 22243_. [doi:10.1038/s41598-020-79148-7](https://doi.org/10.1038/s41598-020-79148-7) - She, Jun; Muñiz Piniella, Ángel; Benedetti-Cecchi, Lisandro; Boehme, Lars; Boero, Ferdinando; Christensen, Asbjorn; Crowe, Tasman; Darecki, Miroslaw; et al., (2019), **An integrated approach to coastal and biological observations**, _Frontiers in Marine Science 6, 314_. [link](https://www.frontiersin.org/articles/10.3389/fmars.2019.00314/full) - Oppenheimer, Michael; Hinkel, Jochen, (2019), **Sea Level Rise and Implications for Low Lying Islands, Coasts and Communities Supplementary Material**. [link](https://www.ipcc.ch/site/assets/uploads/sites/3/2019/11/SROCC_FinalDraft_Chapter4-SM.pdf) - Farcy, Patrick; Durand, Dominique; Charria, Guillaume; Painting, Suzanne J.; Tamminen, Timo; Collingridge, Kate; Grémare, Antoine J.; Delauney, Laurent; et al., (2019), **Toward a European coastal observing network to provide better answers to science and to societal challenges; the JERICO research infrastructure**, _Frontiers in Marine Science 6, 529_. [link](https://www.frontiersin.org/articles/10.3389/fmars.2019.00529/full) - Benveniste, Jérôme; Cazenave, Anny; Vignudelli, Stefano; Fenoglio-Marc, Luciana; Shah, Rashmi; Almar, Rafael; Andersen, Ole; Birol, Florence; et al., (2019), **Requirements for a coastal hazards observing system**, _Frontiers in Marine Science 6, 348_. [link](https://www.frontiersin.org/articles/10.3389/fmars.2019.00348/full) - Kirwan, Matthew L.; Gedan, Keryn B., (2019), **Sea-level driven land conversion and the formation of ghost forests**, _Nature Climate Change 9(6), 450–457_. [link](https://www.nature.com/articles/s41558-019-0488-7) - Ponte, Rui M.; Carson, Mark; Cirano, Mauro; Domingues, Catia M.; Jevrejeva, Svetlana; Marcos, Marta; Mitchum, Gary; Van De Wal, R. S. W.; et al., (2019), **Towards comprehensive observing and modeling systems for monitoring and predicting regional to coastal sea level**, _Frontiers in Marine Science 6, 437_. [link](https://www.frontiersin.org/articles/10.3389/fmars.2019.00437/full) - Stammer, Detlef; Bracco, Annalisa; AchutaRao, Krishna; Beal, Lisa; Bindoff, Nathaniel L.; Braconnot, Pascale; Cai, Wenju; Chen, Dake; et al., (2019), **Ocean climate observing requirements in support of climate research and climate information**, _Frontiers in Marine Science 6, 444_. [link](https://www.frontiersin.org/articles/10.3389/fmars.2019.00444/full) - She, Jun; Muñiz Piniella, Ángel; Benedetti-Cecchi, Lisandro; Boehme, Lars; Boero, Ferdinando; Christensen, Asbjorn; Crowe, Tasman; Darecki, Miroslaw; et al., (2019), **An integrated approach to coastal and biological observations**, _Frontiers in Marine Science 6, 314_. [link](https://www.frontiersin.org/articles/10.3389/fmars.2019.00314/full) - Marbach-Ad, Gili; Hunt, Carly; Thompson, Katerina V., (2019), **Exploring the Values Undergraduate Students Attribute to Cross-disciplinary Skills Needed for the Workplace: an Analysis of Five STEM Disciplines**, _Journal of Science Education and Technology 28(5), 452-469_. [doi:10.1007/s10956-019-09778-8](https://doi.org/10.1007/s10956-019-09778-8) - Reichstein, Markus; Camps-Valls, Gustau; Stevens, Bjorn; Jung, Martin; Denzler, Joachim; Carvalhais, Nuno; Prabhat, (2019), **Deep learning and process understanding for data-driven Earth system science**, _Nature 566(7743), 195-204_. [doi:10.1038/s41586-019-0912-1](https://doi.org/10.1038/s41586-019-0912-1) - Benish, Sarah, (2018), **Meeting STEM workforce demands by diversifying STEM**, _Journal of Science Policy and Governance 13(1), 1-6_. [link](https://www.sciencepolicyjournal.org/uploads/5/4/3/4/5434385/benish.pdf) - Alessa, Lilian; Moon, Sean; Griffith, David; Kliskey, Andrew, (2018), **Operator driven policy: Deriving action from data using the quadrant enabled Delphi (QED) method**, _Homeland Security Affairs 14_. [link](https://www.hsdl.org/c/view?docid=816669) - Sweet, W.; Dusek, G.; March, D.; Carbin, G.; Marra, J., (2018), **State of High Tide Flooding with a 2019 Outlook**, _NOAA Technical Report NOS CO-OPS 90_. - Chabbi, Abad; Loescher, Henry W., (2017), **The lack of alignment among environmental research infrastructures may impede scientific opportunities**, _Challenges 8(2), 18_. [link](https://www.mdpi.com/2078-1547/8/2/18) - Arkema, Katie K.; Verutes, Gregory M.; Wood, Spencer A.; Clarke-Samuels, Chantalle; Rosado, Samir; Canto, Maritza; Rosenthal, Amy; Ruckelshaus, Mary; et al., (2015), **Embedding ecosystem services in coastal planning leads to better outcomes for people and nature**, _Proceedings of the National Academy of Sciences 112(24), 7390-7395_. [doi:10.1073/pnas.1406483112](https://doi.org/10.1073/pnas.1406483112) - Hewlett, Sylvia Ann; Marshall, Melinda; Sherbin, Laura, (2013), **How diversity can drive innovation**, _Harvard business review 91(12), 30–30_. [link](https://www.cs.jhu.edu/~misha/DIReadingSeminar/Papers/Hewlett14.pdf) - Cyranoski, David; Gilbert, Natasha; Ledford, Heidi; Nayar, Anjali; Yahia, Mohammed, (2011), **Education: The PhD factory**, _Nature 472(7343), 276-279_. [doi:10.1038/472276a](https://doi.org/10.1038/472276a) - Titus, James G.; Hudgens, Daniel E.; Trescott, Daniel L.; Craghan, Michael; Nuckols, William H.; Hershner, Carl H.; Kassakian, J. M.; Linn, Chris J.; et al., (2009), **State and local governments plan for development of most land vulnerable to rising sea level along the US Atlantic coast**, _Environmental Research Letters 4(4), 044008_. [link](https://iopscience.iop.org/article/10.1088/1748-9326/4/4/044008/meta) - Nicholls, Robert J.; Wong, Poh Poh; Burkett, Virginia; Codignotto, Jorge; Hay, John; McLean, Roger; Ragoonaden, Sachooda; Woodroffe, Colin D.; et al., (2007), **Coastal systems and low-lying areas**. [link](https://ro.uow.edu.au/scipapers/164/) - Martínez, M. L.; Intralawan, A.; Vázquez, G.; Pérez-Maqueo, O.; Sutton, P.; Landgrave, R., (2007), **The coasts of our world: Ecological, economic and social importance**, _Ecological Economics 63(2), 254-272_. [doi:10.1016/j.ecolecon.2006.10.022](https://doi.org/10.1016/j.ecolecon.2006.10.022) - De Battisti, Davide, (09/2021), **The resilience of coastal ecosystems: A functional trait‐based perspective**, _Journal of Ecology 109(9), 3133-3146_. [doi:10.1111/1365-2745.13641](https://doi.org/10.1111/1365-2745.13641) - Corburn, Jason, (06/2003), **Bringing Local Knowledge into Environmental Decision Making: Improving Urban Planning for Communities at Risk**, _Journal of Planning Education and Research 22(4), 420-433_. [doi:10.1177/0739456X03022004008](https://doi.org/10.1177/0739456X03022004008) - Loescher, Henry W.; Vargas, Rodrigo; Mirtl, Michael; Morris, Beryl; Pauw, Johan; Yu, Xiubo; Kutsch, Werner; Mabee, Paula; et al., (05/2022), **Building a Global Ecosystem Research Infrastructure to Address Global Grand Challenges for Macrosystem Ecology**, _Earth's Future 10(5), e2020EF001696_. [doi:10.1029/2020EF001696](https://doi.org/10.1029/2020EF001696) - Elwood, Sarah A, (05/2002), **GIS Use in Community Planning: A Multidimensional Analysis of Empowerment**, _Environment and Planning A: Economy and Space 34(5), 905-922_. [doi:10.1068/a34117](https://doi.org/10.1068/a34117) - Halpern, Benjamin S.; Boettiger, Carl; Dietze, Michael C.; Gephart, Jessica A.; Gonzalez, Patrick; Grimm, Nancy B.; Groffman, Peter M.; Gurevitch, Jessica; et al., (01/2023), **Priorities for synthesis research in ecology and environmental science**, _Ecosphere 14(1), e4342_. [doi:10.1002/ecs2.4342](https://doi.org/10.1002/ecs2.4342) - **National Strategy for a Sustainable Ocean Economy**. ## Books (8) - Sciences, National Academies of; Earth, Division on; Studies, Life; Sciences, Board on Earth; Sciences (CORES), Committee on Catalyzing Opportunities for Research in the Earth; Sciences, A. Decadal Survey for NSFâ\neg" s Division of Earth, (2020), **A vision for NSF Earth sciences 2020-2030: Earth in time**, _National Academies Press_. [link](https://books.google.com/books?hl=en&lr=&id=NIf4DwAAQBAJ&oi=fnd&pg=PR1&dq=A+Vision+for+NSF+Earth+Sciences+2020-2030:+Earth+in+Time.+&ots=iKoHBty2Z9&sig=uyYXDavKJNTXUq-bPTCusrpyDSI) - Sciences, National Academies of; Behavioral, Division of; Sciences, Social; Earth, Division on; Studies, Life; Change, Board on Environmental; Board, Ocean Studies; Sciences, Board on Earth; et al., (2018), **Understanding the long-term evolution of the coupled natural-human coastal system: the future of the US Gulf Coast**, _National Academies Press_. [link](https://books.google.com/books?hl=en&lr=&id=TQ5zDwAAQBAJ&oi=fnd&pg=PR1&dq=Understanding+the+Long-Term+Evolution+of+the+Coupled+Natural-Human+Coastal+System:+The+Future+of+the+U.S.+Gulf+Coast&ots=Idka8-0nm7&sig=DHsQe4qHo-ovPKbYqOU8ZFRYPS8) - Committee on the Decadal Survey for Earth Science and Applications from Space; Space Studies Board; Division on Engineering and Physical Sciences; National Academies of Sciences, Engineering, and Medicine, (2018), **Thriving on Our Changing Planet: A Decadal Strategy for Earth Observation from Space**, _National Academies Press_. [doi:10.17226/24938](https://doi.org/10.17226/24938) - Scharmer, C. Otto, (2016), **Theory U: Leading from the future as it emerges**, _Berrett-Koehler Publishers_. [link](https://books.google.com/books?hl=en&lr=&id=vZDxCwAAQBAJ&oi=fnd&pg=PP1&dq=Theory+U:+Leading+from+the+future+as+it+emerges.+&ots=RHHG5Zya1x&sig=a0AUx2l7xSpLs59vu5E6Z_8bHR0) - National Research Council, (2015), **Sea Change: 2015-2025 Decadal Survey of Ocean Sciences**, _National Academies Press_. [doi:10.17226/21655](https://doi.org/10.17226/21655) - (2014), **Understanding the Connections Between Coastal Waters and Ocean Ecosystem Services and Human Health: Workshop Summary**, _National Academies Press_. [doi:10.17226/18552](https://doi.org/10.17226/18552) - Senge, Peter M., (2005), **Presence: An exploration of profound change in people, organizations, and society**, _Crown Business_. [link](https://books.google.com/books?hl=en&lr=&id=ms6MDQAAQBAJ&oi=fnd&pg=PA5&dq=Presence:+an+exploration+of+profound+change+in+people,+organizations,+and+society&ots=4fjgI8Msxa&sig=oNTbCzYF4iGPlTB4J7c__9tqVxw) - **Read "Sea-Level Rise for the Coasts of California, Oregon, and Washington: Past, Present, and Future" at NAP.edu**. [doi:10.17226/13389](https://doi.org/10.17226/13389) ## Reports & technical documents (10) - US White House, (June 3, 2024), **White House Briefing: New Strategies to Advance Sustainable Ocean Management**. [link](https://www.whitehouse.gov/ostp/news-updates/2024/06/03/white-house-releases-new-strategies-to-advance-sustainable-ocean-management/) - US White House, (2024), **National Strategy for A Sustainable Ocean Economy: A report by the Ocean Policy Committee**. [link](https://www.whitehouse.gov/wp-content/uploads/2024/06/National-Stategy-for-a-Sustainable-Ocean-Economy_Final.pdf) - United Nations Environment Programme, (2022), **For people and planet: the UNEP strategy for 2022–2025**. [link](https://www.unep.org/resources/people-and-planet-unep-strategy-2022-2025) - US White House, (2022), **National Security Memorandum on Combating Illegal, Unreported, and Unregulated Fishing and Associated Labor Abuses**. [link](https://www.whitehouse.gov/briefing-room/presidential-actions/2022/06/27/memorandum-on-combating-illegal-unreported-and-unregulated-fishing-and-associated-labor-abuses/) - National Science Board (NSB), (2020), **Vision 2030: Vision for the Future. National Science Board Pub**. [link](https://www.nsf.gov/nsb/publications/2020/nsb202015.pdf) - National Science and Technology Council (NSTC), (2020), **Earth System Predictability Research and Development Strategic Framework and Roadmap: A Report by the Fast Track Action Committee on Earth System Predictability Research and Development**. [link](https://trumpwhitehouse.archives.gov/wp-content/uploads/2020/11/Earth-System-Predictability-Research-and-Development-Strategic-Framwork-and-Roadmap.pdf) - National Science and Technology Council (NSTC), (2019), **2019 National Plan for Civil Earth Observations: a Report by the U.S. Group on Earth Observations Subcommittee, Committee on the Environment of the National Science and Technology Council**. - US White House, (2018), **Ocean Policy To Advance the Economic, Security, and Environmental Interests of the United States**. [link](https://www.federalregister.gov/documents/2018/06/22/2018-13640/ocean-policy-to-advance-the-economic-security-and-environmental-interests-of-the-united-states) - Sweet, William V.; Kopp, Robert E.; Weaver, Christopher P.; Obeysekera, Jayantha; Horton, Radley M.; Thieler, E. Robert; Zervas, Chris, (2017), **Global and regional sea level rise scenarios for the United States**. [link](https://ntrs.nasa.gov/citations/20180001857) - President’s Council of Advisors on Science and Technology, (2011), **Sustaining Environmental Capital: Protecting Society and the Economy. Report to the President**. [link](https://www.whitehouse.gov/ostp/pcast) ## Webpages & online resources (21) - (2024), **Digital Coast**. [link](https://coast.noaa.gov/digitalcoast/) - (2024), **NOAA Office for Coastal Management**. [link](https://coast.noaa.gov/) - (2024), **White House Releases New Strategies to Advance Sustainable Ocean Management | OSTP**. [link](https://www.whitehouse.gov/ostp/news-updates/2024/06/03/white-house-releases-new-strategies-to-advance-sustainable-ocean-management/) - House, The White, (2022), **Memorandum on Combating Illegal, Unreported, and Unregulated Fishing and Associated Labor Abuses**. [link](https://www.whitehouse.gov/briefing-room/presidential-actions/2022/06/27/memorandum-on-combating-illegal-unreported-and-unregulated-fishing-and-associated-labor-abuses/) - US White House, (2021), **FACT SHEET: President Biden’s Leaders Summit on Climate**. [link](https://www.whitehouse.gov/briefing-room/statements-releases/2021/04/23/fact-sheet-president-bidens-leaders-summit-on-climate/) - (2018), **Ocean Policy To Advance the Economic, Security, and Environmental Interests of the United States**. [link](https://www.federalregister.gov/documents/2018/06/22/2018-13640/ocean-policy-to-advance-the-economic-security-and-environmental-interests-of-the-united-states) - **Frontiers | An Integrated Approach to Coastal and Biological Observations**. [link](https://www.frontiersin.org/journals/marine-science/articles/10.3389/fmars.2019.00314/full) - **LightCast Data for Technicians**. [link](https://lightcast.io/products/data/overview) - **Education: The PhD factory - Pacific Northwest National Laboratory**. [link](https://pnnl.primo.exlibrisgroup.com) - **Frontiers | Requirements for a Coastal Hazards Observing System**. [link](https://www.frontiersin.org/journals/marine-science/articles/10.3389/fmars.2019.00348/full) - **Frontiers | An Integrated Approach to Coastal and Biological Observations**. [link](https://www.frontiersin.org/journals/marine-science/articles/10.3389/fmars.2019.00314/full) - **Priorities for synthesis research in ecology and environmental science - Halpern - 2023 - Ecosphere - Wiley Online Library**. [link](https://esajournals.onlinelibrary.wiley.com/doi/10.1002/ecs2.4342) - **Redirecting**. [link](https://linkinghub.elsevier.com/retrieve/pii/S0921800906005465) - **Fast Facts**. [link](https://coast.noaa.gov/coastal-facts/) - **Download: Thriving on Our Changing Planet: A Decadal Strategy for Earth Observation from Space | The National Academies Press**. [link](https://nap.nationalacademies.org/download/24938) - **New Report Identifies Three Critical Areas of Research to Fill Gaps in Scientific Knowledge of the Gulf Coasts Interconnected Natural and Human System | National Academies**. [link](https://www.nationalacademies.org/news/2018/06/new-report-identifies-three-critical-areas-of-research-to-fill-gaps-in-scientific-knowledge-of-the-gulf-coasts-interconnected-natural-and-human-system) - **Long-term Coastal Zone Dynamics Interactions and Feedbacks between Natural and Human Processes along the US Gulf Coast | National Academies**. [link](https://www.nationalacademies.org/our-work/long-term-coastal-zone-dynamics-interactions-and-feedbacks-between-natural-and-human-processes-along-the-us-gulf-coast) - **Managed Retreat in the US Gulf Coast Region | National Academies**. [link](https://www.nationalacademies.org/our-work/managed-retreat-in-the-us-gulf-coast-region) - **Representing the function and sensitivity of coastal interfaces in Earth system models | Nature Communications**. [link](https://www.nature.com/articles/s41467-020-16236-2) - **Coastal Wetlands in the Anthropocene | Annual Reviews**. [link](https://www.annualreviews.org/content/journals/10.1146/annurev-environ-121922-041109) - **Feedbacks Regulating the Salinization of Coastal Landscapes | Annual Reviews**. [link](https://www.annualreviews.org/content/journals/10.1146/annurev-marine-070924-031447) ## Notes (1) - . ------------------------------------------------------------------------ ## Reference documents report URL: https://tyson-swetnam.github.io/cod-kmap/docs/reference_documents_report.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/reference_documents_report.md # Reference documents — what we read and what we kept The Coastal Observatory Design (COD) team has compiled a substantial body of background material — coastal-zone definitions, prior observing-network surveys, NSF Mid-scale guidance, organisational diagrams. This page summarises what is in that reference set and how each piece informs cod-kmap. ## Source material | Group | What it contains | |---|---| | **Coastal definition documents** | The Coastal Zone matrix, the Coastal Graphic v4.0, and the PCAST Earth-Observation Interoperability v2.2 — together they define the coastal critical zone we are scoping. | | **2018 Coastal Observatory Landscape survey** | 32 community responses listing every coastal observing network the community knew about, with measured systems, scale, scope, and US / international reach. The seed for the cod-kmap `networks` and `facilities` tables. | | **MSI handouts (2022, 2024, 2026)** | Successive proposal drafts; show how the pitch evolved over four years. | | **NSF references** | Mid-scale R2 solicitations NSF 19-542 and NSF 21-537, the PAPPG, the EVMS Gold Card, and the Research Infrastructure Guide NSF 21-107. They define the funding mechanism and reporting rules cod-kmap aligns with. | | **Reference images and diagrams** | Finkl 2004 Coastal Classification, NAS 2020 *Environmental Science in the Coastal Zone*, the Six-Sigma Network → Research Infrastructure transition diagram, and the Requirements Flowdown image. They give us a formal coastal taxonomy and the design lifecycle. | | **WATERS Network documents (2006–2009)** | The full Science, Education, and Cyberinfrastructure plans for WATERS, the previous attempted national aquatic observatory that was never built. Useful as a lessons-learned source. | | **Borer et al. 2020** (BioScience) and **Sci. Adv. 2022** | Two flagship papers on integrated coastal observation and on the critical-zone framing. | ## Organisational structure > **Superseded, 2026-07-26.** The summary below describes the Clemson > CCZO **2024** chart. The 2026 revision reorganises the work-breakdown > structure into seven tracks and changes several assignments — Workforce > and Broader Impacts is now 3.0 under K. Lazar, Cyberinfrastructure is > 5.0 under Tyson Swetnam, Prototype Infrastructure is 6.0, and the > Project Management Office is 7.0. The current chart is transcribed in > full in `data/seed/cod_wbs.csv` and `data/seed/cod_team_members.csv`, > rendered by the **Team** tab, and its provenance is documented in > [Team, Scholars & Data Methods](#/docs/team-scholars-datasets-methods). > The 2024 text is kept here as a record of how the project was > originally structured. The Clemson CCZO 2024 organisational chart and the Design Flow diagram together describe a five-track work-breakdown structure: - **1.0 COD Management & Support** — PI Skip Van Bloem (Clemson) + Deputy Hank Loescher (Battelle), with governance led by S. Whitmire. - **2.0 Science Management** — Chief Scientist Allison Myers-Pigg (PNNL), with six Integrated Project Teams: Physical Environment, Social Dimension, Biotic Environment, Integrative Design, Coastal Informatics, and Built Environment. - **3.0 Broader Impacts Management** — led by Christie Staudhammer, with a STEM Workforce pipeline integrating NEON, OBFS, and Clemson programmes. - **4.0 Cyberinfrastructure** — co-CIs Nirav Merchant and Tyson Swetnam (University of Arizona), covering data ingest, processing, discovery, and the workbench design. - **5.0 Prototype Infrastructure** — Engineering Manager K. Nitschke. - **6.0 Project Management Office** — C. Ritz, PMP. External community partners named in the chart include LTER, NEON, OOI, WHOI, Scripps, RDA, EDI, AGU Biogeochem, EREN, USGCRP working groups (IWGs and SOCCR), NOAA Earth System Research Lab, NASA Coastal, Environmental Defense Fund, Woodwell Climate Research Center, and the State Climate Action Alliance. ## Design lifecycle The Design Flow document describes an iterative cycle: 1. The Science Leadership Committee and Science IPTs distil the Grand Challenge questions into specific data and product needs: what we have, what is missing, where the spatial and forecast gaps are. 2. The Cyberinfrastructure team captures, integrates, and harmonises coastal datasets across time, space, and discipline, with AI/ML support for synthesis. 3. A tailored AI environment with semantic ontologies and controlled vocabularies provides web services, open-source software, and ISO-compliant DOIs and provenance tracking. 4. A public data portal exposes the integrated dataset with Jupyter notebooks, R libraries, Docker, GitHub, CodeLabs, and Python packages. 5. The Draft Coastal Observatory Design goes through public community review, final concurrence, and acceptance, producing an implementation-ready package (project execution plan, total construction cost budget, resource-loaded baseline schedule, staffing plans, risk register, ConOps, decommissioning plans, and the Critical Design Review). The Design Flow names the existing curated coastal datasets the observatory should integrate: MarineGEO, NERR, LTER, OOI, Neptune-Coastal, CODISS, Coastal Change Analysis Program, Coastal Zone Management, Digital Coast, SCCOOS, the Coastal Data Information Program (CDIP), the Sea Level Change Portal, the National Coastal Condition Assessment, and the Integrated Ocean Observing System (IOOS). ## What the 2018 survey changed in cod-kmap The 2018 community survey listed 29 distinct observing networks. Cross-referencing against cod-kmap's 32-network catalogue: - **Already represented:** IOOS, SECOORA, LTER, NERR, USGS streamgauging, NDBC, MarineGEO, NWLON, CBIBS, NCCA, NPS coastal. - **Added or scheduled:** AmeriFlux, the Coastwide Reference Monitoring System (CRMS, Louisiana), Gulf of Maine Ocean Observing System, the National Atmospheric Deposition Program, the Gulf of Mexico Hypoxia Watch, GEO-BON, the Mexican IMECOCAL, the Colombian REDCAM, Germany's COSYNA, Chesapeake Bay water and SAV monitoring, and the Coastal Carolina Nearshore monitor network. ## How the reference set drives the data model The reference materials inform several extensions to the cod-kmap schema, listed here in priority order. ### A coastal-zone strata table The Coastal Zone matrix and Finkl 2004 classification define the coastal critical zone along multiple axes. cod-kmap captures these as a per-facility table: ```sql CREATE TABLE coastal_zone_strata ( facility_id VARCHAR REFERENCES facilities(facility_id) PRIMARY KEY, zone VARCHAR, -- littoral, sublittoral, inner-shelf, mid-shelf, outer-shelf, abyssal shore_type VARCHAR, -- rocky, sandy, muddy, deltaic, mangrove, coral, kelp, tidal-flat, salt-marsh river_status VARCHAR, -- riverine, estuarine, tidal-river, none built_status VARCHAR, -- urban, suburban, rural, natural ocean_basin VARCHAR, -- N-Atlantic, S-Atlantic, N-Pacific, S-Pacific, Arctic, Caribbean, Gulf-of-Mexico, Great-Lakes, Hudson-Bay finkl_class VARCHAR -- one of Finkl 2004's 21 coastal classes ); ``` ### IPT mapping for research areas Each research area is tagged with its corresponding Integrated Project Team — Physical Environment, Biotic Environment, Coastal Informatics, Social Dimension, Integrative Design, or Built Environment — so the Stats and Network views can filter by IPT. ### Curated coastal datasets — **implemented** The 14 datasets named in the Design Flow are tracked in a dedicated `coastal_datasets` table and linked back to the facilities that contribute. This lets cod-kmap show "which COD partner stewards this dataset?" alongside the facility view. > **Built, 2026-07-26**, and expanded well past the original 14: the > catalogue holds 82 datasets with 258 access endpoints, including all 11 > IOOS regional associations and the federal archives. A companion > `dataset_endpoints` table carries the ERDDAP / THREDDS / OPeNDAP / OGC / > REST / S3 / STAC endpoints, which is what makes the catalogue usable > rather than merely descriptive. Rendered by the **Data** tab. The > facility linkage is via `network_id`; a direct dataset-to-facility > stewardship edge is still future work. ### Partner organisations A `partner_organisations` table catalogues the named external partners (universities, networks, agencies, NGOs, working groups) with their COD role: science leadership, data source, workforce pipeline, or governance. ### Grand Challenge framing The four Grand Challenge questions (Challenging Theory, Societal Responses, Coastal Vulnerability, Uncertainties in Coastal Ecosystem Processes) are embedded in the Stats and Network views: each research area surfaces which Grand Challenge it contributes to. ### COD team in the People directory — **implemented, with changes** The ~30 named people in the organisational chart appear in the researcher directory with their COD role explicitly tagged, and are filterable via a "COD Team" facet in the People view. > **Built, 2026-07-26**, as its own **Team** tab rather than a facet on > the People view. The org chart is a hierarchy — seven WBS tracks, a > leadership committee, and unfilled positions — and a flat filtered card > grid could not show that structure. The 40 named members are still > synced into `people`, so they appear in the researcher directory too and > the existing OpenAlex/ORCID enrichment scripts pick them up. > > A second, larger roster came out of the same work: `community_scholars` > holds 523 coastal-ocean-science researchers across the pre-eminent, > most-active, and rising cohorts, rendered by the **Scholars** tab. That > one is deliberately a separate table from `people` — it is a > field-wide bibliometric cohort, not the staff of catalogued facilities. > > **Updated 2026-07-27.** Being separate tables no longer means being > unlinked. `person_registry` resolves the org chart, the facility staff > directory and the scholar roster into one identity space keyed on a > persistent identifier, so a person who belongs to more than one can be > recognised as one human and a co-publication graph can be computed > across all three at once. The original ask — the team visible alongside > the wider community, with the connections between them legible — is > satisfied by that table rather than by a facet on the People view. See > [The Person Registry](#/docs/person-registry). ## Lessons from WATERS The WATERS Network was the prior attempt at a national aquatic observatory. Its 2009 Science Plan was reviewed by the National Research Council, and the network was never built — primarily because of cost overruns and scope creep. Two design choices in COD explicitly avoid that path: - **AI/ML and cyberinfrastructure are core, not bolt-on.** WATERS treated cyberinfrastructure as a downstream dependency; COD treats it as Track 4 with its own work-breakdown structure. - **Phased, distributed prototyping** rather than a single large-scale build. Each Integrated Project Team can deliver a working prototype before the full observatory commits to a final design. ## What the reference set explicitly leaves alone cod-kmap's existing taxonomies — research areas, networks, facility types — are kept as authored. The IPT mapping and other extensions above are additive, not replacements. The OpenAlex and publication- topic crosswalks are independent of the proposal-document review and remain unchanged. ------------------------------------------------------------------------ ## Map visualization plan URL: https://tyson-swetnam.github.io/cod-kmap/docs/map_visualization_plan.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/map_visualization_plan.md # Knowledge map — visualization design The **Network** tab in cod-kmap renders the dataset as a country-style map: each research area is a polygon, polygon area is proportional to the number of facilities working in that area, and facilities and researchers sit inside the polygon they belong to. Edges between polygons reveal interdisciplinary collaboration. This page documents the design choices and the algorithm behind the view. ## What the layout shows A **Map Visualization with Group restriction (MVG)** treats categorical groups in a network as polygonal regions: - Each group gets exactly one polygon. - Polygon area is proportional to group size (facility count). - Polygon adjacency reflects connectivity between groups. - Node positions inside a polygon reflect intra-group structure plus pull from external collaborations. The algorithm follows Hossain, Moradi, Mondal & Kobourov, *Map Visualizations for Graphs with Group Restrictions*, Graphics Interface 2025 ([DOI 10.1145/3769872.3769900](https://doi.org/10.1145/3769872.3769900)). A reference implementation by the same University of Arizona group powers [kmap.arizona.edu](https://kmap.arizona.edu). ## Choice of grouping Researchers and facilities have multiple categorical attributes. We chose **research area** (35 polygons) as the default grouping for three reasons: - It is the most semantically meaningful grouping for science users. - We have weighted assignments per person and facility from publication topics. - Polygon sizes vary naturally — *marine ecosystems* dominates, *Great Lakes* and *tsunamis-and-coastal-hazards* are small — producing a visually informative cartogram. Other dimensions are exposed as a grouping toggle: | Grouping | Polygons | Notes | |-----------------|---------:|-------| | Research area | 35 | Default. Most analytically meaningful. | | Network | 32 | Single-membership; clean borders. | | Facility type | 10 | Fewer polygons; useful for high-level summary. | | Funding agency | ~10 | Surfaces the funding landscape per facility. | ## Algorithm The layout is computed in three steps: 1. **Supergraph embedding.** Build one super-node per group (weight = number of nodes in the group, edge weight = number of cross-group links). Embed with a force layout, then pack each super-node as a square sized by the square root of its weight. 2. **Per-group subgraph layout.** Run an independent force simulation on the nodes inside each group, then scale and translate to fit inside its square. 3. **Polygon partitioning.** Compute a Voronoi diagram over all nodes and merge cells that share a group, producing one polygon per group. Smooth the boundaries with a single Chaikin pass. A higher-fidelity refinement (PCL — Polygon-Constrained Layout) adds boundary-aware central gravity, corner gravity, and external gravity toward adjacent polygons. PCL produces fewer edge crossings but takes ~10× longer to converge; we expose it as a power-user toggle. ## Interaction - **Click a polygon** to zoom in and dim the others; a panel shows the group name, member count, top funders, top researchers, and total funding. - **Click a node** to highlight all its edges (intra-group dark, cross-group light) and open the facility or researcher card. - **Filter chips** (country, type, area subset, funder subset) hide matching nodes while preserving the polygon shapes — so changing a filter doesn't re-jiggle the layout. ## Performance The full graph (~700 nodes when every toggle is on) renders in under a second with the default fast layout, then upgrades to PCL in the background. Polygon fills and node-link edges render to a ``; selection / hover state and polygon labels render to SVG on top. Switching back to a previous grouping reads from a per-grouping cache and is instant. ## Open design questions - Should the layout collapse small sub-areas into their parent area (e.g. *salt-marshes* + *tidal-wetlands* under *estuaries-and- wetlands*) when the per-area facility count is below a threshold? The current view keeps all 35 areas; collapsing reduces clutter at the cost of taxonomic detail. - Should filter-driven hiding shrink the polygon for the filtered group, or grey it out at proportional opacity? Today: grey it out. ------------------------------------------------------------------------ ## Network fix metrics URL: https://tyson-swetnam.github.io/cod-kmap/docs/NETWORK_FIX_METRICS.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/NETWORK_FIX_METRICS.md # network_fix_metrics.csv M1-M7 (Hossain, Moradi, Mondal & Kobourov, GI'25 S5.3) for the COD knowledge map before and after 59 researchers move out of facility circles into interstitial space. Real graph: 443 nodes, 289 edges (209 within-group, 80 between-group), 21 groups. ## The M7 crossing decomposition M7 = 1 - (crossing edge PAIRS) / |E|^2. Every pair of edges is classified by whether each edge is within-group or between-group, so the three counts sum to the total and reproduce M7 exactly: | component | before | after | delta | share of increase | |---|---:|---:|---:|---:| | within x within | 0 | 2 | +2 | 2% | | within x between | 22 | 131 | +109 | 86% | | between x between | 672 | 688 | +16 | 13% | | **total** | **694** | **821** | **+127** | | M7 0.9917 -> 0.9902. **Correction.** Earlier versions of this table, the commit message for d4218ad, and my report to the user all stated that the M7 loss was "within-group only" with "between-group crossings unchanged". That was wrong twice over: * the fraction of between-group edges involved in a crossing ROSE, 0.10500 -> 0.10750 -- 50x the within-group change in absolute terms; * the dominant term is within x between (86% of the increase). The two single-subset fractions I originally reported never summed to the M7 change at all, because cross-subset pairs were not measured. Moving nodes toward polygon boundaries puts them nearer the between-group lines that leave the polygon, so those lines now cross more intra-area edges. That is the cost of the fill improvement (M6 over groups with >=3 nodes: 0.0916 -> 0.1644); it is not free, and it lands partly on exactly the between-group structure the map is read for. ## Fidelity caveat This is a MODEL of the change, not the shipped layout. Polygons come from `mvg_kmap` on the real graph, NOT from network.js's own layoutSupergraph + computePolygons (which need d3's force simulation and a browser). The 59 moved nodes are displaced outward toward their polygon boundary to approximate `interstitialSlots()`; per-area move COUNTS are exact, individual coordinates are not. Direction and rough magnitude are evidence; absolute values are not what the site will produce. ------------------------------------------------------------------------ ## Funding pipeline plan URL: https://tyson-swetnam.github.io/cod-kmap/docs/funding_pipeline_plan.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/funding_pipeline_plan.md # Funding data — sources and methods cod-kmap aims to record every meaningful source of funding for each of the ~210 coastal facilities in the dataset — federal, state, private, foundation, charitable — for fiscal years **2015 through 2024**, with per-award and per-fiscal-year granularity. Every funding record carries a confidence label (high / medium / low) and a verifiable source URL. ## What's in the database Funding flows through three tables: - **`funders`** — one row per granting body (NSF, NOAA, EPA, USGS, NIH, Walton Family Foundation, etc.) with type, country, and URL. - **`funding_events`** — one row per (funder, facility, award, fiscal year) with the dollar amount, award identifier, programme name, reporting source, source URL, and confidence rating. - **`funding_links`** — a backwards-compatible view over `funding_events` for older queries. Confidence ratings are applied at row-write time: - **High** — primary federal source (NSF Awards API, USAspending.gov) or audited financial filing (Form 990). - **Medium** — agency budget book line items, transcribed by hand from a Congressional Justification PDF or annual report. - **Low** — press-release dollar amounts, infographic figures, or derivative summaries. ## Data sources ### Federal grant-making (already loaded) **NSF Awards API.** Hits `api.nsf.gov/services/v1/awards.json` per facility, filtered by quoted-phrase awardee name and (optionally) programme name or keyword. Each award's per-fiscal-year obligation list becomes one event row — no estimation, exact figures from NSF. Coverage today: 12 facilities, 261 events, $133.4M. Some facilities still need awardee-name verification before NSF returns matches — typically university-hosted marine labs whose awards are billed under the parent campus, not the lab itself. ### Federal grant-making (next) **USAspending.gov** covers every federal grant *outside* NSF — NOAA, EPA, DOI/USGS, DOD, DOE, NIH. Records include award ID, recipient name, award amount, description, awarding agency, programme code, and period of performance. Expected coverage: roughly 75 of the 210 facilities, with overlap against the NSF set. ### Non-profit financial filings **ProPublica Nonprofit Explorer (Form 990).** Covers the 33 US non-profit facilities (Hakai, Dauphin Island Sea Lab, Mote Marine Laboratory, etc.) plus the 3 foundations. Each Form 990 reveals total revenue, total expenses, net assets per fiscal year, and (when disclosed) major contributors and grants disbursed. ### Agency-internal allocations Most of the 76 US federal facility units don't appear in USAspending: they are intramural NOAA, EPA, NPS, USGS, or USACE programmes funded via line-item appropriations. Examples include the Channel Islands National Marine Sanctuary's annual operating budget, Long Island Sound Study NEP's EPA award, and NPS coastal-park resource-management allocations. We populate these from agency budget books: - NOAA NOS budget rollouts (annual Congressional Justification PDFs). - EPA NEP funding history. - NPS Green Book (annual Congressional budget request). - USGS budget justifications. ### State, foundation, and non-US A long tail of smaller programmes covers state marine labs (8 facilities), international institutes (DFO Canada, Mexican and Latin American institutes, Hakai BC — ~15 facilities), and private foundations whose 990s aren't on ProPublica. Most of these are populated by hand against agency websites in the relevant language. ## Validation Before publishing each funding-data update, we cross-check against the published agency-programme totals to confirm the rows roll up correctly: - NSF LTER programme: ~$28M / year - EPA NEP total annual budget: ~$30M (28 NEPs ≈ $1M each) - NOAA Sea Grant national network: ~$80M / year split across 33 programmes - ONR / DARPA / NASA earth-science marine awards: ~$50M / year combined A separate sanity check ensures no single facility sums to more than $500M / decade — only the largest ocean institutes plausibly exceed that. ## Where the data lives in the UI The **Network** tab uses funder identity as one of the optional groupings. Each facility's card lists its funders with cumulative amounts. The **Stats** tab shows the per-research-area funding distribution. The **SQL** tab exposes the raw `funding_events` table for ad-hoc analysis. ------------------------------------------------------------------------ ## Suitability roadmap URL: https://tyson-swetnam.github.io/cod-kmap/docs/suitability_roadmap.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/suitability_roadmap.md # Site suitability — a roadmap for "where should the next observatory go?" cod-kmap currently answers the question *where do existing coastal research facilities sit?*. To answer the harder question — *where would a new observatory be most representative of the coastal critical zone?* — three additional data layers and a ranking algorithm are needed. This page describes the layers and the ranking method as a roadmap. The site-suitability tab in the application becomes available once all three layers are loaded. ## Three data layers ### Geographic and climatic strata What the world looks like, by ocean ecoregion and climate zone. | Source | Scale | License | |--------|------:|---------| | **MEOW** (Marine Ecoregions of the World, Spalding et al. 2007) | 232 marine ecoregions | CC-BY | | **Köppen-Geiger** climate zones (Beck et al. 2018) | 30 classes, 1-km raster | CC-BY | | **GADM** coastal admin boundaries | 38,000+ polygons world-wide | free for non-commercial | | **EEZ** boundaries (Marine Regions Flanders v12) | 281 EEZs | CC-BY | The corresponding cod-kmap tables hold per-facility membership in every stratum: ``` facility_strata facility_id → facilities.facility_id meow_ecoregion → meow_ecoregions.ecoregion_id koppen_class → e.g. 'Cfa', 'Dfb' eez_id → eez_zones.eez_id gadm_admin1 → state or province ``` ### Human-influence proxies Is this place still wild? | Source | Scale | License | |--------|------:|---------| | **GHSL Population** (JRC, 2023) | 100-m raster, global | CC-BY | | **Cumulative Human Impact** (Halpern 2019) | 1-km raster, global oceans | CC-BY | | **WCMC Marine Protected Areas** | 17,000+ MPAs | CC-BY | | **Distance to nearest major port** | derived from the World Port Index | computed | ``` facility_human_influence pop_density_per_km2 -- GHSL within 5 km of HQ cumulative_impact -- WCMC 0-15 score inside_mpa -- HQ inside any WDPA polygon distance_to_port_km -- nearest WPI port influence_score_0_to_1 -- composite (0 = pristine, 1 = heavy pressure) ``` ### Biological diversity proxies Is this place biologically rich? | Source | Scale | License | |--------|------:|---------| | **GBIF** marine occurrence records | global | CC-BY | | **OBIS** Ocean Biogeographic Information System | global oceans | CC0 | | **MARSPEC** marine spatial environmental indices | 1-km raster | CC-BY | | **Reef Life Survey** species lists | global reefs | CC-BY | Per-facility metrics, computed within a 50-km buffer: ``` facility_diversity gbif_n_species, gbif_n_records obis_n_species marspec_temp_range, marspec_salinity_var diversity_score_0_to_1 ``` ## The ranking algorithm Once all three layers are loaded, the candidate-site ranker tiles the coastline into 50-km hexagonal cells (H3 resolution 4) and scores each cell against each research area: ``` score = (stratum_emptiness × 0.4) + (diversity_score × 0.3) + ((1 - influence) × 0.2) + (representativeness × 0.1) ``` - **Stratum emptiness** captures how under-represented a stratum is for a given research area: `1 − (existing_facilities / max_facilities_per_stratum)`. - **Diversity** rewards biologically rich cells. - **Inverse influence** rewards relatively pristine cells. - **Representativeness** rewards cells that are statistically distinctive within their region. Cells within 100 km of an existing same-area facility are excluded so the ranker doesn't recommend "another observatory next to this observatory." The top 25 cells per research area form the published candidate list. ## Where it shows up in the UI A new card in the per-area dashboard: | Location | Ecoregion | Diversity | Pristine | Score | |---|---|---|---|---| | Pribilof Islands, AK | Aleutian Islands | 0.91 | 0.86 | 0.87 | | (next four) | … | … | … | … | Each row links into the Map tab, pre-zoomed to the candidate cell with a proposed-site marker. ## Effort and dependencies The three ingestion passes plus the ranking algorithm and dashboard add up to roughly six and a half working days: | Phase | Days | |---|---:| | Geographic and climatic strata (MEOW, Köppen, EEZ, GADM) | 1.5 | | Human-influence raster sampling (GHSL, WCMC) | 1.0 | | Diversity ingestion (GBIF, OBIS) — many API calls, async | 2.0 | | Hex-cell tiling and stratum-emptiness scoring | 1.0 | | Suitability ranking and CSV export | 0.5 | | Dashboard card and map drill-in | 0.5 | | **Total** | **6.5** | Software dependencies: - DuckDB ≥ 1.5 with the spatial extension. - Python: `rasterio`, `geopandas`, `h3`, `shapely`, `requests`. The largest downloads are GHSL Population at 100 m (~6 GB) and the WDPA shapefile (~1 GB). Köppen-Geiger is ~200 MB; MEOW and EEZ are small. ## Why the suitability layer isn't shipping in the first release It's substantial, requires several large file downloads, and the ranking should be reviewed by domain experts before being exposed in the UI as "go build an observatory here." The current dashboard already gives operators a strong "what coverage exists today?" view; the suitability layer is a logical next step rather than a prerequisite. ------------------------------------------------------------------------ ## Personnel gap research URL: https://tyson-swetnam.github.io/cod-kmap/docs/personnel_gap_research_plan.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/personnel_gap_research_plan.md # Researcher and personnel coverage Each facility in cod-kmap is associated with the people who lead it — typically a Director, Reserve Manager, Principal Investigator, Research Coordinator, or equivalent. Every personnel record carries a verifiable source URL and a confidence rating, and is loaded only when the source can be cited. ## Coverage today Of the 210 facilities in the catalogue, **roughly 95 % have at least one named, verified leader recorded** in `facility_personnel`. The remaining ~5 % are intentionally left without a leader entry: their public-facing pages do not list a named individual (some sentinel sites, some virtual networks). We do not fabricate leadership records. A NULL is more honest than a guess, and the audit log captures every facility we couldn't resolve along with the page we read. ## Source priorities by facility group | Group | Source | |---|---| | NOAA NERR reserves | The reserve's own website (Manager + Research Coordinator pages); the NERRA member directory cross-references all 30. | | EPA NEPs | The NEP's own website; EPA's NEP programme page links every Director. | | NSF LTREB sites | NSF Award Search by LTREB programme and site keyword returns the Principal Investigator directly. | | Fisheries and Oceans Canada institutes | Each institute's "About Us" page on dfo-mpo.gc.ca; the Government of Canada GEDS directory confirms incumbent. | | IOOS Regional Associations | Each Regional Association's "About / Leadership" page. | | Latin American institutes | CICESE, INIDEP, INAPESCA, ICML-UNAM directorate pages (Spanish-language). | | Latin American and Caribbean universities | CIMAR (Costa Rica), IO-USP (Brazil), UWI-CERMES (Barbados), UWI-PRML (Jamaica) — directorate pages, mixed Spanish, Portuguese, and English. | | US federal headquarters and singletons | Agency director pages (NCCOS, GLERL, NRL-SSC, FRF, NASA Goddard OBPG, etc.). | ## Record fields Each row in `facility_personnel` captures: - `person_id` and `facility_id` (foreign keys). - `role` and `title` — e.g. *Director*, *Lead Principal Investigator*, with the title verbatim from the source page. - `is_key_personnel` — true for Directors, Reserve Managers, Research Coordinators, and equivalent leadership roles. - `start_date` and `end_date` where the source page or appointment notice gives them. - `source` and `source_url` — the page we cited. - `confidence` — `high` (named on a programme page or press release), `medium` (named on a partner page or news article), or `low` (inferred from a directory listing without a recent date). - `notes` — a short free-text qualifier (incumbent date, transition status, etc.). ## When a record changes Facility leadership changes regularly (NERR managers, EPA Regional Administrators, NEP Directors, university lab heads). Each personnel-research pass: - Re-checks every existing record's source URL. - Captures incumbents with start dates where available. - Marks superseded records with an `end_date` rather than deleting them, so the audit history is preserved. ## How the data surfaces - **People** tab — every researcher card shows their facility roles with title, country, and a link to the facility's homepage. - **Network** tab — the per-facility tooltip lists the named Director / PI(s). - **SQL** tab — the `v_facility_key_personnel` view exposes the current key personnel for any facility in a single row. ------------------------------------------------------------------------ ## ORCID enrichment URL: https://tyson-swetnam.github.io/cod-kmap/docs/orcid_enrichment_plan.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/orcid_enrichment_plan.md # ORCID-based researcher enrichment cod-kmap links each researcher to their ORCID identifier wherever one is available. ORCID is the international standard for persistent, self-claimed researcher identifiers, and we use it as the primary disambiguator before linking to OpenAlex, Scopus, or Google Scholar. ## Why ORCID is the anchor ORCID identifiers are claimed by the researcher themselves. Once a person's ORCID is linked, every downstream lookup becomes deterministic: - **OpenAlex**: `/authors?filter=orcid:0000-...` returns exactly one author record. - **Scopus** and **Web of Science**: ORCID linkage is published in the ORCID record's *external identifiers* section. - **Publisher metadata**: increasingly required by journals, so newer publications are reliably attributed. Without ORCID, downstream resolvers fall back to name search, which collapses any two researchers who happen to share a name. ## Resolution rules A candidate ORCID is accepted only when **all** of the following hold: - Family name matches exactly (case-insensitive, diacritic-normalised). - The first given name matches (handles "Sarah" vs. "Sarah J." vs. "Sarah Jane"). - The candidate's current or past employments include an organisation that shares at least one **distinctive token** with one of our facility records for that person — a proper noun or domain word, with generic organisation vocabulary ("research", "university", "national", "institute") stripped out first — and then clears the score threshold. - If multiple candidates pass the above, prefer the most recent employment, then the candidate already linked to one of our OpenAlex authors. The distinctive-token requirement is a gate, not a scoring term, and it exists because character-level similarity between two normalised organisation names sits in the same band for unrelated organisations as it does for the same one. An earlier version scored candidates on character similarity alone and accepted several employer-to-facility matches between organisations with nothing in common beyond shared letters. The character-level score is now averaged in at low weight rather than being able to carry a match by itself. Where only a name matches and no employment can be tested, acceptance requires that the name resolve to exactly **one** distinct ORCID. Two or more candidates are logged as ambiguous and left null — an earlier version promised this in a comment but in fact took the first candidate after a sort in which every candidate scored zero, which for common names meant picking arbitrarily among dozens of people. If no candidate satisfies every rule, no ORCID is recorded. A NULL identifier is preferable to a wrong one. ## Sources and rate limits The ORCID Public API is free and requires no key: - `GET https://pub.orcid.org/v3.0/expanded-search/?q=` — candidate profiles with name, current employments, and external identifiers (rate-limit ~24 req/s). - `GET https://pub.orcid.org/v3.0//employments` — confirm a candidate against our facility records. ## Coverage Coverage in the facility-staff directory is low: **49 of 280** `people` rows carry a verified ORCID. It was briefly higher, before an audit found identifiers already in the committed data that pointed at different people entirely and cleared them; the strict matcher above then declined to re-resolve most of those names. That is the intended trade. The un-linked rows are typically: - Reserve managers, programme directors, and similar administrative roles whose work doesn't appear in indexed journals. - Researchers who haven't claimed an ORCID record yet. - Researchers whose name is shared with enough other people that no candidate can be distinguished safely. These rows stay un-linked rather than risk a wrong attribution. A periodic re-run picks up newly-claimed ORCIDs without manual work. ORCID coverage is far higher in `person_registry`, which draws identifiers from author records rather than resolving names: 86,620 of 152,008 identities are keyed on an ORCID, 9,309 of the 10,000 rows served to the browser. Those are ORCIDs asserted by the bibliographic source on an author record, not resolutions this repository performed. See [The Person Registry](#/docs/person-registry). ## Audit trail Every resolution decision (accept / reject / no candidate) is logged to `data/seed/orcid_resolution_log.csv` with the candidate ORCID, similarity scores, and reason. The log is the source of truth for "why does this person not have an ORCID?" questions. ------------------------------------------------------------------------ ## Google Scholar enrichment URL: https://tyson-swetnam.github.io/cod-kmap/docs/google_scholar_enrichment_plan.md Raw source: https://raw.githubusercontent.com/tyson-swetnam/cod-kmap/main/docs/google_scholar_enrichment_plan.md # Google Scholar profile linkage Where a researcher has a public Google Scholar profile, cod-kmap surfaces a direct link on the researcher card. Scholar's h-index and citation history are the most widely-recognised academic metrics, and the link is high-value for non-specialist readers. Google Scholar has no official API. We populate the field via a tiered approach, preferring deterministic sources over scraping. ## Tiered sources | Tier | Source | Method | Measured coverage | |-----:|-------------------|---------------------------------------|---------:| | 1 | OpenAlex | `ids.scholar` field | ~0% | | 2 | ORCID | `external-identifiers` block | ~0% | | 3 | Institutional homepage | Stored in `people.homepage_url`; reader follows the link | indirect | | 4 | Paid SerpAPI / scholar_author | JSON; reserved for high-value queries | not run | **The first two tiers have now been run, and they return almost nothing.** The projected coverage in an earlier version of this page — roughly half the directory — was an estimate, and it was wrong by two orders of magnitude. Measured results: | Table | Rows | With a Scholar id | |---|---:|---:| | `people` (facility staff) | 280 | 4 | | `community_scholars` (roster) | 523 | 12 | | `person_registry` (unified) | 152,008 | 12 | OpenAlex does not populate `ids.scholar` for the overwhelming majority of author records, and ORCID's external-identifiers block rarely carries one either. This is a property of the upstream sources, not a defect in the scripts: the field the tiers read is simply empty. The 12 ids in `person_registry` were all carried in from seed data rather than resolved, but not all from the team seed as this page previously claimed: 2 came from `people` (Myers-Pigg and Swetnam) and 10 from `community_scholars`. Anything approaching useful coverage would require a different source or a hand-curation pass. Until then, treat a missing Scholar link as the default state rather than as a gap to be explained. ## Schema ``` people.google_scholar_id : VARCHAR community_scholars.google_scholar_id : VARCHAR person_registry.google_scholar_id : VARCHAR ``` Format: the `user_id` segment of the Scholar URL, e.g. `xKqqKf4AAAAJ` for `https://scholar.google.com/citations?user=xKqqKf4AAAAJ`. ## Front-end When `google_scholar_id` is present, the researcher card adds a **Google Scholar** link beside the homepage and ORCID buttons. Otherwise the card silently omits the link rather than showing a broken affordance. ## Why we don't scrape Scholar by default The community `scholarly` Python package can scrape Scholar pages, but Google rotates anti-bot measures every few months. Any pipeline built on `scholarly` becomes operationally fragile and requires occasional configuration changes. We've chosen to defer this work until there's a clear product reason to need 100% Scholar coverage.