Skip to content

Add functionality to create centroids and centroid connector to model network - #478

Open
lmz wants to merge 285 commits into
network-wrangler:mainfrom
BayAreaMetro:centroids
Open

lmz wants to merge 285 commits into
network-wrangler:mainfrom
BayAreaMetro:centroids

Conversation

@lmz

@lmz lmz commented Jun 19, 2026

Copy link
Copy Markdown
Member

This PR adds end-to-end support for creating centroids and centroid connectors in a model network, including the core APIs, geometry/link generation logic, and tests/documentation updates needed to use and validate the workflow.

What this PR does

This PR introduces functionality to build model-zone access structures by generating:

  • Centroid nodes (zone representation in the network)
  • Centroid connector links between centroids and nearby network nodes
  • The feature is designed to support common modeling workflows where zones/TAZs need to be integrated into the routable network with generated access/egress links.

Key changes

  • Added APIs/utilities to create centroid nodes from input zone geometry/data.
  • Added logic to create centroid connector links from each centroid to eligible nearby network nodes.
  • Added validation and guardrails around connector creation inputs and outputs.
  • Added/updated tests covering centroid and connector creation behavior.
  • Updated docs/examples to demonstrate expected usage and outputs.

Why this change is needed

Many demand and assignment workflows require zone-level connectors to represent how trips enter/exit the physical network. This PR provides a native, repeatable way to generate those structures directly in network_wrangler, reducing manual preprocessing and ensuring consistent connector generation behavior.

Implementation notes

Centroid generation and connector generation are implemented as explicit network-building operations so they can be run as part of a reproducible wrangling pipeline.
Connector creation is based on proximity/eligibility rules to attach centroids to nearby nodes.
The new functionality is intended to be configurable for model-specific connector policies.

Testing

This PR includes tests for:

  • Correct centroid creation from valid inputs
  • Connector generation for expected nearest/eligible nodes
  • Error/edge handling for invalid or missing inputs
  • Basic integrity checks on produced nodes/links
  • Backward compatibility

Existing workflows are unaffected unless the new centroid/connector functionality is explicitly invoked. No breaking changes are intended for users not using these new APIs.

Limitations / follow-ups

Connector selection policies may still need tuning for specific model conventions (e.g., max connectors per zone, facility/type filtering, directional behavior).
Future enhancements can expand configurability and performance for very large zone sets.


lmz added 30 commits July 25, 2025 19:02
Also trying to simplify code and create fewer copies of things
…s; inherit this classification from their children
lmz added 14 commits August 3, 2026 13:21
Replace per-row .apply() with GeoSeries/Series numpy operations (addresses review comments on get_link_bearing_degrees and bearing_to_cardinal_direction).

- get_link_bearing_degrees -> get_link_bearings_degrees(geometry: GeoSeries): extracts
  endpoints with shapely vectorized get_point/get_x/get_y and computes forward azimuths
  in one pyproj Geod.inv call; NaN where geometry is missing/empty/degenerate.
- bearing_to_cardinal_direction -> bearings_to_cardinal_directions(bearings: Series):
  numpy sector floor-divide against _CARDINAL_LABELS/_INTERCARDINAL_LABELS; None where NaN.
- add_direction_to_links now calls the vectorized helpers instead of two nested applies.
Replace the per-node modal-graph iteration with a vectorized groupby over
modal links (mode_query): out-degree via size() and node fitness via max()
of the incident links' *_centroid_fit. Import all node attributes through a
single inner merge on nodes_df instead of a hardcoded column list, dropping
the street_count dependency (which does not exist on every network) and the
redundant fillna/astype cleanup.

Addresses review comments L211, L236, L249.
Add tests covering add_centroid_nodes, add_centroid_connectors, and
calculate_angle_from_centroid against the small example network. The
connectors test deliberately omits street_count on the nodes so it guards
the mode_query/groupby node evaluation, and asserts per-zone connector
counts (not specific node choices) so it remains valid across future
selection-loop refactors.
- Vectorize centroid distance and angle-bearing calculations in centroid connector selection.
- Replace apply-heavy per-zone connector picking with incremental min-angle separation updates.
- Refactor ML_geometry normalization into a dedicated helper to reduce full-column coercion work.
- Harden parquet writes by adding a fallback that JSON-encodes nested object columns when Arrow conversion fails.
- Add regression test coverage for parquet nested-object fallback behavior.
@e-lo

e-lo commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

I see you are hard at work on this - LMK when you are ready for another review :-)

@lmz

lmz commented Aug 27, 2026

Copy link
Copy Markdown
Member Author

I see you are hard at work on this - LMK when you are ready for another review :-)

I think I've addressed your comments and it's ready for another look!
Thank you!! (And I'm still pushing hard on contracting...)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants