From cab1a5695ecf29da82deaf9e41fc70e094d4c1a5 Mon Sep 17 00:00:00 2001 From: David Kane Date: Thu, 2 Jul 2026 10:12:41 -0400 Subject: [PATCH] student: add mapping/census stack (sf + tidycensus) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bake sf and tidycensus (which pulls in tigris) into the student image so census-tract mapping works out of the box — base already ships the GDAL/PROJ/GEOS/udunits system libs for exactly this. Smoke test loads all three and runs an offline st_sample() on a CRS-tagged polygon to verify the geospatial linkage end to end. Interactive maps use the already-baked leaflet over free CARTO tiles (no Mapbox token needed). Also update the commit-trailer convention to name the current model. Co-Authored-By: Claude Fable 5 --- CLAUDE.md | 3 ++- README.md | 1 + student/Dockerfile | 11 +++++++++++ 3 files changed, 14 insertions(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index 698f3e9..162f553 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -76,6 +76,7 @@ Contents (in addition to `base`): These same four are **also** refreshed to latest on every Codespace create by codespace-starter's `postCreateCommand`. Baking them here pre-installs the dependency tree (so that refresh is a quick update, not a cold build) and is a fallback if GitHub is unreachable. Add further course packages to **both** places (here and the postCreateCommand) following the same pattern. - A basic **Python data-science stack** so students can work in Python alongside R (numpy, pandas, matplotlib, seaborn, scikit-learn, statsmodels, jupyter, ipykernel). Installed with `uv` into a venv at `/opt/venv` from a fully-pinned lockfile — `student/requirements.lock`, generated from `student/requirements.txt` via `uv pip compile` (edit the `.txt` and recompile to bump). Notable choices, all build-test-verified: the venv uses the **system** CPython (`uv venv --python /usr/bin/python3`, not a managed download); it's the default `python` on `PATH`, `--seed`ed (has its own `pip`), and made **group-writable** (`staff`, like R's site-library) so a student can `pip install` more packages. A `python3` Jupyter kernel ("Python (data science)") is registered system-wide, so Python works in VS Code notebooks and in Quarto `.qmd` chunks — Quarto runs Python-only docs via the **jupyter** engine, so **reticulate is not needed** (and isn't installed; `RETICULATE_PYTHON` is still set so reticulate bridges to this venv if someone does install it). Footprint: ~150 MB compressed / negligible Codespace-creation time (baked into the image, restored by the prebuild). - **Observable** for interactive JavaScript visualisation. Quarto's bundled `{ojs}` cells render Observable Plot / OJS inside `.qmd` documents (pass R/Python data in via `ojs_define()`) — this needs **nothing extra**, it's part of the baked Quarto (build-test verified). Separately, **Observable Framework** (`@observablehq/framework`, pinned; adds the `observable` CLI for `create`/`preview`/`build`) is installed for building standalone data-app/dashboard projects — ~84 MB on disk, smoke-tested via `observable --version`. +- A **mapping / census stack** for NYT-style tract maps: `sf` (vector geodata — links against the GDAL/PROJ/GEOS/udunits system libs that `base` bakes for exactly this) and `tidycensus` (ACS/decennial data + geometry in one call; pulls in `tigris`). Live `tidycensus` queries need a free per-student Census API key. Interactive maps come free from the pieces already here: `leaflet` draws `sf` objects, with CARTO basemap tiles instead of Mapbox (no token). The smoke test does an offline `st_sample()` on a CRS-tagged polygon to verify the GDAL/PROJ/GEOS linkage end to end. - **Interactive web-viz + Shinylive**, for "fancy" interactive websites that still publish to **static** hosting (GitHub Pages). R htmlwidgets — `plotly`, `leaflet`, `DT`, `crosstalk` (installed via pak, ~70 MB) — and their Python counterparts `plotly`/`altair`/`folium`/`itables` (in `requirements.lock`) emit self-contained client-side JS. `shiny` + `shinylive` (both R and Python) compile Shiny apps to WebAssembly (webR/Pyodide) so even reactive apps run with no server. (Interactive charts *inside* Quarto docs work via these widgets or `{ojs}`; Shinylive embeds via the per-project `quarto-shinylive` extension.) Does NOT include: @@ -141,7 +142,7 @@ The steps to take when changing this infrastructure. `devcontainers` (this repo) ### Conventions -- Commit trailer on every commit: `Co-Authored-By: Claude Opus 4.8 (1M context) `. +- Commit trailer on every commit, naming whichever Claude model did the work, e.g.: `Co-Authored-By: Claude Fable 5 `. - Never pin student-facing config to `:latest`. - Report build/prebuild results honestly (status + conclusion), and clean up throwaway diagnostic branches/workflows when done. diff --git a/README.md b/README.md index 0d759b1..c7cd4d0 100644 --- a/README.md +++ b/README.md @@ -34,6 +34,7 @@ The `student` image carries both languages students use: - **R** — the rocker/tidyverse foundation, plus the PPBDS course packages (`tutorial.helpers`, `vscode.tutorials`, `misc.tutorials`, `primer.tutorials`), the `arf` console, Quarto, and `httpgd`. - **Python** — a basic data-science stack installed with [`uv`](https://docs.astral.sh/uv/) into `/opt/venv` from a pinned lockfile (`student/requirements.lock`): numpy, pandas, matplotlib, seaborn, scikit-learn, statsmodels, jupyter, ipykernel. It's the default `python` on `PATH`, with a registered Jupyter kernel — so students can use Python in VS Code notebooks or in Quarto `.qmd` chunks (Quarto runs Python via the jupyter engine; no reticulate needed). The venv is group-writable, so students can `pip install` more packages. - **Observable** — Quarto's `{ojs}` cells render interactive [Observable Plot](https://observablehq.com/plot/) / OJS in `.qmd` documents out of the box (no install); plus the [Observable Framework](https://observablehq.com/framework/) CLI (`observable`) for building standalone data-app projects. +- **Mapping / census** — `sf` + `tidycensus` (with `tigris`) for census-tract maps: pull ACS data and geometry in one call, render static maps with `ggplot2` or interactive ones with `leaflet` over free CARTO basemap tiles (no Mapbox token). Live census queries need a [free Census API key](https://api.census.gov/data/key_signup.html). - **Interactive web viz + Shinylive** — for fancy interactive sites on static hosting: R htmlwidgets (`plotly`, `leaflet`, `DT`, `crosstalk`) and Python (`plotly`, `altair`, `folium`, `itables`) emit self-contained client-side JS; **Shinylive** (`shiny` + `shinylive`, R & Python) runs Shiny apps as WebAssembly, so they too deploy to GitHub Pages with no server. Both are baked into the image, so they cost ~nothing at Codespace-creation time (the prebuild restores them). diff --git a/student/Dockerfile b/student/Dockerfile index 88de6c8..5d3c106 100644 --- a/student/Dockerfile +++ b/student/Dockerfile @@ -75,6 +75,17 @@ RUN R --vanilla -e 'install.packages("praise", repos = "https://cloud.r-project. RUN R -q -e 'pak::pkg_install(c("plotly", "leaflet", "DT", "crosstalk", "shiny", "shinylive"))' \ && R --vanilla -e 'for (p in c("plotly","leaflet","DT","crosstalk","shiny","shinylive")) if (!requireNamespace(p, quietly = TRUE)) stop("smoke test failed to load: ", p)' +# Mapping / census stack — NYT-style tract maps (dot-density, choropleths): +# sf (vector geodata; links against the GDAL/PROJ/GEOS/udunits system libs that +# base bakes for exactly this) and tidycensus (ACS/decennial data + geometry in +# one call; pulls in tigris for TIGER shapefiles). Students need a free Census +# API key for live tidycensus queries. P3M Linux binaries, so this is fast. +# The interactive path needs nothing more: leaflet (above) draws sf objects, +# and free CARTO basemap tiles substitute for Mapbox. +RUN R -q -e 'pak::pkg_install(c("sf", "tidycensus"))' \ + && R --vanilla -e 'for (p in c("sf","tidycensus","tigris")) if (!requireNamespace(p, quietly = TRUE)) stop("smoke test failed to load: ", p)' \ + && R --vanilla -e 'library(sf); p <- st_sfc(st_polygon(list(rbind(c(0,0), c(1,0), c(1,1), c(0,1), c(0,0)))), crs = 4326); pts <- st_sample(p, 10); stopifnot(length(pts) == 10); cat("sf geometry ops OK\n")' + USER root # ── Python data-science stack ────────────────────────────────────────────────