Production-harden loader, multi-package dependency tracking, exhaustive tests

.Rbuildignore

@@ -1,2 +1,12 @@
 ^renv$
 ^renv\.lock$
+^\.Rprofile$
+^\.github$
+^\.gitignore$
+^test_loadfast\.R$
+^devpackage$
+^AGENTS\.md$
+^TECHNICAL_DEBT\.md$
+^justfile$
+^pkgload$
+^devtools$

.github/workflows/ci.yml

@@ -8,17 +8,54 @@ on:
 
 jobs:
   test:
+    name: harness (${{ matrix.os }})
     runs-on: ${{ matrix.os }}
 
     strategy:
       fail-fast: false
       matrix:
-        os: [ubuntu-latest, windows-latest]
+        os: [ubuntu-latest, windows-latest, macos-latest]
+
+    env:
+      RENV_CONFIG_AUTOLOADER_ENABLED: "FALSE"
 
     steps:
       - uses: actions/checkout@v4
       - uses: r-lib/actions/setup-r@v2
         with:
-          r-version: "4.5.2"
-      - uses: r-lib/actions/setup-renv@v2
+          r-version: "release"
+          use-public-rspm: true
+      # Installs Imports + Suggests from DESCRIPTION (everything the harness
+      # needs) and handles system requirements; the renv.lock is for local dev.
+      - uses: r-lib/actions/setup-r-dependencies@v2
       - run: Rscript test_loadfast.R
+
+  check:
+    name: R CMD check (${{ matrix.os }}, R ${{ matrix.r }})
+    runs-on: ${{ matrix.os }}
+
+    strategy:
+      fail-fast: false
+      matrix:
+        os: [ubuntu-latest, windows-latest, macos-latest]
+        r: ["release"]
+        include:
+          - os: ubuntu-latest
+            r: "oldrel-1"
+
+    env:
+      RENV_CONFIG_AUTOLOADER_ENABLED: "FALSE"
+
+    steps:
+      - uses: actions/checkout@v4
+      - uses: r-lib/actions/setup-r@v2
+        with:
+          r-version: ${{ matrix.r }}
+          use-public-rspm: true
+      - uses: r-lib/actions/setup-r-dependencies@v2
+        with:
+          extra-packages: any::rcmdcheck
+          needs: check
+      - uses: r-lib/actions/check-r-package@v2
+        with:
+          args: 'c("--no-manual", "--as-cran")'

.gitignore

@@ -1,2 +1,4 @@
 pkgload/
 devtools/
+loadfast_*.tar.gz
+loadfast.Rcheck/

AGENTS.md

@@ -12,9 +12,9 @@ This `AGENT.md` file is read by every agent session. !!!Keep them high-signal!!!
 ## Project layout
 
 - This repo **is** the source-of-truth edit target for the `loadfast` package. If you are changing loader behavior, docs, or tests for `loadfast`, make the change here.
-- **`R/loadfast.R`** contains the canonical loader implementation exported by the package. It provides MD5-based incremental reloading: on first call it does a full teardown+rebuild; on subsequent calls for the same path it re-sources only files whose MD5 hash changed. See "Incremental loader" section below.
-- **`DESCRIPTION`**, **`NAMESPACE`**, and **`inst/rstudio/addins.dcf`** define the installable package and its RStudio addin registration.
-- **`test_loadfast.R`** is the single custom test runner. It sources `R/loadfast.R` directly and remains the single command users run.
+- **`R/loadfast.R`** contains the canonical loader implementation exported by the package. It provides MD5-based incremental reloading: on first call it does a full teardown+rebuild; on subsequent calls for the same path it re-sources only files whose MD5 hash changed. The public `load_fast()` is a thin re-entrance guard around the internal `.loadfast.load()`, which can recurse to reload flagged dependencies. See "Incremental loader" section below.
+- **`DESCRIPTION`**, **`NAMESPACE`**, and **`inst/rstudio/addins.dcf`** define the installable package and its RStudio addin registration. `NAMESPACE` and `man/` are **generated by roxygen2** (`roxygen2::roxygenize(".")`) — edit the roxygen comments in `R/loadfast.R`, not the generated files. Dev/test-only packages live in `Suggests`.
+- **`test_loadfast.R`** is the main custom test runner (~300 checks, stages 1-7). It sources `R/loadfast.R` directly and remains the single command users run. **`tests/testthat/`** additionally holds a small self-contained suite that runs against the *installed* package during `R CMD check`.
 - **`devpackage/`** is the single frozen baseline package snapshot used by the tests. Contains `DESCRIPTION`, `NAMESPACE`, `R/` (source files), and `tests/testthat/` (testthat tests + helpers). Package name is `devpackage`. All code mutations for reload/incremental testing are applied ad-hoc to temp copies at test time.
   - `R/base.R` — plain functions (`add`, `scale_vector`, `summarize_values`, `mutate_dt`) — `mutate_dt` exercises `data.table`'s `:=` and `as.data.table` via `importFrom`
   - `R/s3_classes.R` — S3 constructor (`new_temperature`), a package-defined generic (`describe_s3`), and methods on both base generics (`print`/`format`/`as.character`) and the local generic. Registered via `S3method()` in NAMESPACE, none exported by name — so dispatch depends on the loader populating the S3 methods table.
@@ -31,7 +31,7 @@ This `AGENT.md` file is read by every agent session. !!!Keep them high-signal!!!
 
 - The package name is **read from the `Package:` field in DESCRIPTION**. No hard-coding required — just set the field in your DESCRIPTION file.
 - `load_fast(path)` accepts a package root or any path inside a package. It walks upward until it finds a directory containing `DESCRIPTION`, `NAMESPACE`, and `R/`.
-- Requires **`rlang`** as a runtime dependency (for `rlang::ns_registry_env()`). The user already has it because they use devtools.
+- No third-party runtime dependencies: the namespace registry is accessed via runtime-constructed `.Internal(registerNamespace(...))` / `.Internal(unregisterNamespace(...))` calls (`.loadfast.ns_register()` / `.loadfast.ns_unregister()`), the same approach current pkgload uses. Do NOT use `rlang::ns_registry_env()` -- it is defunct as of rlang 1.2.0 and hard-errors.
 - DESCRIPTION is parsed only for the `Package:` field. Imports are read from the **NAMESPACE** file.
 - `load_fast()` accepts `helpers` and `attach_testthat` parameters (mirroring pkgload's `load_all`). When `helpers = TRUE` (the default) and testthat is available, it calls `testthat::source_test_helpers()` to source `helper*.R` files from `tests/testthat/` into the attached package environment — the same behavior as pkgload.
 - `attach_testthat` defaults to auto-detect: if `tests/testthat/` (or `inst/tests/`) exists and testthat is installed, testthat is attached to the search path via `library("testthat")`.
@@ -40,7 +40,11 @@ This `AGENT.md` file is read by every agent session. !!!Keep them high-signal!!!
 
 ## Incremental loader (`R/loadfast.R`)
 
-- **`.loadfast.cache`** is a module-level environment (`parent = emptyenv()`) keyed by `normalizePath(path)`. Each entry currently stores the namespace env, file hashes, `renv.lock` hash baseline, any registered reload files, and the pending reload message to emit on the next load.
+- **`.loadfast.cache`** is a module-level environment (`parent = emptyenv()`) keyed by `normalizePath(path)`. Each entry stores the namespace env, `pkg_name`, file hashes, `renv.lock` hash baseline, the S3 methods matrix, `import_pkgs` (names of packages this package imports), `needs_full` (NULL, or a reason string set by dependency invalidation), any registered reload files, and the pending reload message to emit on the next load.
+- **Dependency invalidation**: after any load that re-sources files, `.loadfast.invalidate_dependents()` flags every cached package that (transitively) imports the reloaded package by setting `needs_full`. A flagged package's next `load_fast()` is upgraded to a full reload, and the full-load path first recursively reloads flagged *dependencies* (via `.loadfast.load()`, guarded by the `.loadfast.state$stack` path stack), so one call on the top of a chain converges the whole chain. No-change loads do not invalidate.
+- **Depends**: `.loadfast.parse_depends()` reads `Depends` from DESCRIPTION (minus `R (...)`), and the full-load path attaches each with `library()` if not already attached; a missing Depends package is a clear error.
+- **Teardown** (`.loadfast.teardown()`): detach + `unloadNamespace()`. When the namespace is imported by other loaded namespaces, `unloadNamespace()` errors — the fallback runs the old namespace's `.onUnload` best-effort and force-removes it from the registry via `.loadfast.ns_unregister()`. A failed full load cleans up after itself (unregisters the half-built ns, detaches) via an `on.exit` guard.
+- **Package identity**: if the `Package:` field changes in place for a cached path, the old name is detached/unloaded and the cache entry dropped before the full load of the new name.
 - **Change detection**: `tools::md5sum()` on all `R/*.R` files every call. Compared against cached hashes to classify files as changed or added.
 - **No per-file symbol tracking**: the incremental path does **not** track which symbols came from which file, and does **not** remove stale symbols when files are deleted or functions are removed. This avoids the O(n²) `ls()` overhead that dominated load time (27.5s of 33.6s in a 475-file project). Stale symbols linger until the user calls `load_fast(path, full = TRUE)`.
 - **Package env sync**: after incremental re-sourcing, all symbols from `ns_env` are bulk-copied to the `package:pkg` environment (one `ls()` call total).
@@ -64,6 +68,8 @@ This `AGENT.md` file is read by every agent session. !!!Keep them high-signal!!!
     - 3d: add a `Collate` field plus deliberately misordered filenames, then verify the loader respects `Collate` ordering rather than plain alphabetical order.
   - **Stage 4**: Copy `devpackage/` to a third temp dir, test incremental-specific behaviors: no-change short-circuit, function removal (stale symbols linger), function addition, function modification, new file, file deletion, explicit file reload registration, failed incremental reload recovery, runtime S4 method patch reload registration, persistent `renv.lock` change warnings, `.onLoad`/`.onAttach` hook execution (4l), and S3 method table refresh on incremental reload (4m). Verifies that `full = TRUE` properly cleans up stale symbols.
   - **Stage 5**: Build ad-hoc packages (distinct class names, so nothing collides) to test cross-package fidelity: `exportPattern()` populating exports and `pkg::fn` resolving (5a), `pkg::missing` giving a proper not-exported error rather than the `lazydata` internal (5b), S3 dispatch across package boundaries for both base and package-defined generics (5c), and `exportClasses()`/`exportMethods()` + `importClassesFrom()`/`importMethodsFrom()` for S4 (5d).
+  - **Stage 6**: Multi-package dependency invalidation: direct importer auto-upgraded to full reload (6a), transitive chain convergence with recursive dependency reload (6b), no-change loads do not invalidate (6c), whole-namespace `import()` invalidation (6d), S4 class redefinition propagating to importers (6e), and the forced-unregister path when `unloadNamespace()` refuses (including `.onUnload` execution) (6f).
+  - **Stage 7**: Production hardening: `Depends` attachment + missing-Depends error (7a), rename-in-place cleanup (7b), failed-full-load cleanup + recovery (7c), namespace version from DESCRIPTION (7d), Collate order on incremental re-source (7e), re-entrance guard (7f), verbose timing logs (7g).
 - Stage 2 always triggers a full load (different path from Stage 1 = cache miss). Stage 3 exercises the incremental path for `load_fast()` on a stable temp path while mutating files in place. The multi-package stages (3e, 3h) call `strip_s3_fixture()` on their copies so the shared `temperature` S3 fixture from the baseline does not register colliding methods across renamed packages.
 
 ## R namespace machinery gotchas
@@ -74,7 +80,7 @@ S4's `setClass`/`setMethod` rely on `topenv()` and `isNamespace()` internally. A
 2. Set up `.__NAMESPACE__.` metadata environment with `spec`, `exports`, `imports`, etc.
 3. **Register the env in R's internal namespace registry** so `isNamespace()` returns `TRUE`.
 
-R's `registerNamespace` is an `.Internal` and cannot be called from user code. The workaround is `rlang::ns_registry_env()` which returns the registry as an R environment — you can assign directly into it: `reg[[pkg_name]] <- ns_env`.
+R's `registerNamespace`/`unregisterNamespace` wrappers are not exported from base, and `rlang::ns_registry_env()` is defunct as of rlang 1.2.0 (it hard-errors). The working approach -- also used by current pkgload -- is constructing the `.Internal()` call at runtime: `eval(as.call(list(quote(.Internal), as.call(list(quote(registerNamespace), name, env)))))`. See `.loadfast.ns_register()`.
 
 ### `makeNamespace` is not exported
 `makeNamespace` is a **local function defined inside `base::loadNamespace`**. It is not accessible directly. pkgload extracts it at load time via AST manipulation (`extract_lang` + `modify_lang`). In `R/loadfast.R` we instead replicate its logic inline.

DESCRIPTION

@@ -1,6 +1,6 @@
 Package: loadfast
 Title: Fast Incremental Package Loader for Interactive Development
-Version: 0.0.1
+Version: 1.0.0
 Authors@R: person(given = "Finccam", family = "Team", email = "engineering@finccam.com", role = c("aut", "cre"))
 Description: Fast incremental reloading for interactive development on large
     R packages. Intended for edit-reload-test workflows where
@@ -12,9 +12,12 @@ Encoding: UTF-8
 Roxygen: list(markdown = TRUE)
 RoxygenNote: 7.3.2
 Imports:
-    rlang
-DevDeps:
-    testthat,
-    R6,
+    methods,
+    tools
+Suggests:
     data.table,
-    digest
+    digest,
+    pkgload,
+    R6,
+    rlang,
+    testthat

NAMESPACE

@@ -1,2 +1,4 @@
+# Generated by roxygen2: do not edit by hand
+
 export(load_fast)
 export(load_fast_register_reload)