[DRAFT] Send compressed gridmap and costmaps over network

[ErikCald]

The costmaps and gridmaps should be able to compress down, especially because alot of the costmap is uniformly unknown. o I made some nodes that will compress them and send as uint8 arrays over the antennas to hopefully make it lighter on the system and make the data more available for me.

Draft PR because i dont know when I will get a chance to actually test if this works and how much it compresses it down by. Need to test it IRL. I could do it on my jetson orin nano probably.

[Copilot]

Pull request overview

Adds a new ROS 2 ament_python package intended to compress and transport Nav2 costmap and traversability grid map data over a constrained network link (as UInt8MultiArray payloads), with corresponding decompressors for local visualization.

Changes:

• Introduces compressed_telemtry ROS 2 Python package scaffolding (setup/package metadata, license, resource marker, lint tests).
• Adds costmap compressor/decompressor nodes using rclpy.serialization + zlib (including a periodic full-map “heartbeat” resend).
• Adds grid map compressor/decompressor nodes using a custom binary header + zlib and quantized payload encoding for elevation/traversability layers.

Reviewed changes

Copilot reviewed 11 out of 13 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
src/Nav/compressed_telemtry/test/test_pep257.py	Adds ament PEP257 lint test for the new package.
src/Nav/compressed_telemtry/test/test_flake8.py	Adds ament flake8 lint test for the new package.
src/Nav/compressed_telemtry/test/test_copyright.py	Adds copyright lint test (currently skipped).
src/Nav/compressed_telemtry/setup.py	Python package setup + ROS 2 console script registration.
src/Nav/compressed_telemtry/setup.cfg	Installs scripts into the ROS 2 package lib directory.
src/Nav/compressed_telemtry/resource/compressed_telemtry	ament resource index marker file for package discovery.
src/Nav/compressed_telemtry/package.xml	ROS 2 package manifest for the new package.
src/Nav/compressed_telemtry/LICENSE	Adds Apache-2.0 license text for the package.
src/Nav/compressed_telemtry/compressed_telemtry/gridmap_compressor.py	Compresses GridMap into a custom binary payload and publishes it.
src/Nav/compressed_telemtry/compressed_telemtry/gridmap_decompressor.py	Decompresses custom grid map payload back into GridMap for RViz.
src/Nav/compressed_telemtry/compressed_telemtry/costmap_compressor.py	Compresses full costmap + incremental updates with zlib.
src/Nav/compressed_telemtry/compressed_telemtry/costmap_decompressor.py	Decompresses costmap payloads and republishes for RViz.
src/Nav/compressed_telemtry/compressed_telemtry/init.py	Marks the Python package module.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ConnorNeed]

My main concern is how heavy the processing for this would be. Python subscribers are notoriously bad which large data. We can test this weekend :)

[ErikCald]

My main concern is how heavy the processing for this would be. Python subscribers are notoriously bad which large data. We can test this weekend :)

Yeah i probably need to implement in cpp, which also means it can be composable and do shared memory zero copy in order to get the costmaps and gridmap. But this will still be a good proof of concept to see how much compression actually helps or not

[Copilot]

Pull request overview

Copilot reviewed 13 out of 15 changed files in this pull request and generated 10 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[ErikCald]

Compressed full costmap: 250104 bytes -> 725 bytes (99.7% smaller, 0.00x of original)

Compressed gridmap: 4889 bytes -> 1097 bytes (77.6% smaller, 0.22x of original)
Compressed send rate: 0.044 Mbps over last 10s (54999 bytes, avg reduction 77.5%, avg ratio 0.22)

[ErikCald]

Gemini comment:

The problem comes down to missing MultiArrayLayout metadata in the Float32MultiArray objects you are publishing, which crashes the C++ subscriber (RViz) and subsequently locks up the Python publisher.

Here is the exact breakdown of what is happening:

1. Why RViz crashes (Exit Code 139 / Segfault)
   Inside the GridMap message structure, each layer (elevation, traversability) is repesented by a Float32MultiArray. The C++ codebase for grid_map_ros (and the RViz plugin by extension) uses an internal helper function called isRowMajor that checks how the array is structured. It specifically checks message.layout.dim[0].label to see if the first dimension is named "column_index" or "row_index".
   Because your script leaves layout completely empty, dim is an empty array. When C++ tries to access index [0] of that empty array, it results in an out-of-bounds memory access (Segmentation fault), instantly killing RViz.

2. Why the Python node hangs
   When a subscriber (RViz) dies abruptly (segfault) right as the Python node is busy serializing and transmitting a massive GridMap message over the network layer (DDS), the underlying DDS middleware can sometimes get stuck waiting for acknowledgments or holding locks in the publisher. Additionally, calling .tolist() on massive NumPy arrays causes a CPU spike that locks up the Python GIL, making the node feel unresponsive to Ctrl+C.

The Fix:
You need to explicitly construct and attach a MultiArrayLayout to both elev_multi and trav_multi before publishing. You have to specify the two dimensions (MultiArrayDimension) based on the grid shape (length_x / resolution and length_y / resolution) and label them as "column_index" and "row_index".