feat: MuJoCo drone simulation with city environment

[ouazmourad]

• DroneController in dimos/simulation/mujoco/policy.py (PD attitude controller)
• SimulatedDroneConnection in dimos/robot/drone/sim.py (minimal interface module)
• drone_sim_agentic() blueprint in dimos/robot/drone/blueprints/sim/
• MuJocoSkillProxy for UDP command bridge to standalone viewer
• Standalone mujoco_sim.py viewer with keyboard controls
• model.py updated with skydio_x2 robot support and drone assets

[greptile-apps]

Greptile Summary

Adds a full MuJoCo-based drone simulation stack for the Skydio X2 quadrotor in a city environment. The PR introduces:

• DroneController (PD attitude controller) in policy.py — converts velocity commands into 4-motor thrusts via collective thrust, tilt targeting, attitude PD, and yaw lock/rate control
• SimulatedDroneConnection in sim.py — the primary integration module with proper DimOS stream/skill interface matching DroneConnectionModule
• MuJocoSkillProxy — a lightweight UDP bridge so the standalone viewer can receive agent commands
• mujoco_sim.py — standalone interactive viewer with keyboard controls and a DimOSBridge for agent integration
• drone_sim_agentic blueprint — composes the simulated drone with visualization, LLM agent, and web input
• model.py updates — adds skydio_x2 robot type with drone-specific PD controller (instead of ONNX) and asset loading

Key issues found:

• mujoco_sim.py: publish_state is completely dead code — _odom_pub is never assigned, so the agent running alongside the standalone viewer has no access to drone state (position, heading, GPS)
• mujoco_sim.py: takeoff/land commands have no altitude enforcement — takeoff sets a constant upward velocity but never stops ascending; land descends indefinitely with no ground check. Compare with sim.py which correctly implements polling checks
• sim.py: Timer-based _check() closures lack null guards on self._controller — could raise AttributeError if a Timer fires during shutdown after the controller is cleaned up

Confidence Score: 3/5

• The core simulation module (sim.py) and controller (policy.py) are solid, but the standalone viewer (mujoco_sim.py) has functional gaps that prevent full agent integration.
• Score of 3 reflects that the main integration path (SimulatedDroneConnection + blueprint) is well-implemented and follows existing patterns, but the standalone viewer has dead publish_state code and missing altitude enforcement in takeoff/land handlers. These are functional bugs that would affect the user experience when running the standalone viewer mode.
• dimos/robot/drone/mujoco_sim.py has the most issues (dead publish_state, no altitude checks). dimos/robot/drone/sim.py has a minor shutdown safety concern in its Timer callbacks.

Important Files Changed

Filename	Overview
dimos/robot/drone/blueprints/sim/drone_sim.py	Clean blueprint composing SimulatedDroneConnection, visualization, agent, and web input. Follows existing blueprint patterns well.
dimos/robot/drone/mujoco_sim.py	Standalone MuJoCo viewer + DimOS bridge. Has two logic issues: (1) publish_state is dead code since _odom_pub is never set, (2) takeoff/land commands don't enforce altitude/ground checks. Also silently swallows exceptions in command listener.
dimos/robot/drone/mujoco_skill_proxy.py	UDP skill proxy forwarding agent commands to MuJoCo viewer. Clean implementation with proper skill annotations. The draw_dimos sky-writing feature is creative. No critical issues.
dimos/robot/drone/sim.py	Main simulated drone connection module. Well-structured with proper stream/skill interface matching DroneConnectionModule. Has a Timer-based polling pattern in takeoff/land that could crash if controller is cleaned up during shutdown, and hasattr-based timing init is fragile across restarts.
dimos/simulation/mujoco/model.py	Adds skydio_x2 robot support with a special early-return path for drones (PD controller instead of ONNX) and loads drone mesh/texture assets. Clean integration following existing patterns.
dimos/simulation/mujoco/policy.py	Adds DroneController PD attitude controller with velocity commands, yaw lock/rate tracking, and X-config motor mixing. Solid control-theory implementation with reasonable gains and clipping.

Sequence Diagram

sequenceDiagram
    participant User as User / Web UI
    participant Agent as LLM Agent
    participant Proxy as MuJocoSkillProxy
    participant Bridge as DimOSBridge (UDP)
    participant Ctrl as DroneController
    participant Sim as MuJoCo Physics

    User->>Agent: Natural language command
    Agent->>Proxy: @skill move(vx, vy, vz, dur)
    Proxy->>Bridge: UDP JSON {"type":"velocity",...}
    Bridge->>Ctrl: set_velocity(vx, vy, vz, yaw)
    
    loop Physics step (~100Hz)
        Ctrl->>Sim: compute_control() → [t1,t2,t3,t4]
        Sim->>Sim: mj_step(model, data)
    end

    Note over Bridge,Sim: publish_state ~30Hz (currently dead in standalone mode)
    
    alt SimulatedDroneConnection mode
        Sim-->>Agent: odom, gps, status via DimOS streams
    else Standalone viewer mode
        Sim-->>Bridge: state available but _odom_pub=None
    end

Loading

_{Last reviewed commit: e3846f8}

[greptile-apps]

Takeoff altitude is parsed but never enforced

The alt parameter is extracted from the command but the drone just ascends at a constant 1.5 m/s indefinitely — there is no altitude check or timer to stop the ascent. The drone will fly upward forever (or until it leaves the simulation bounds). By contrast, SimulatedDroneConnection.takeoff() in sim.py correctly implements a _check() polling loop that stops ascent when the target altitude is reached.

Similarly, the land handler (line 113-115) sets a constant -0.5 m/s descent with no ground check, so the drone will keep pushing downward after landing.

Consider adding altitude-check logic similar to sim.py:

Suggested change

	elif t == "takeoff":
	alt = float(cmd.get("altitude", 3.0))
	self.controller.set_velocity(0, 0, 1.5, 0)
	print(f"[Agent] takeoff to {alt}m")
	elif t == "takeoff":
	alt = float(cmd.get("altitude", 3.0))
	self.controller.set_velocity(0, 0, 1.5, 0)
	print(f"[Agent] takeoff to {alt}m")
	def _check_alt(target: float = alt) -> None:
	pos = self.controller.data.xpos[self.controller.body_id]
	if pos[2] >= target * 0.9:
	self.controller.set_velocity(0, 0, 0, 0)
	return
	threading.Timer(0.2, _check_alt).start()
	threading.Timer(0.5, _check_alt).start()

[greptile-apps]

publish_state is a dead code path — always returns early

self._odom_pub is initialized to None on line 37 and is never assigned a real value anywhere in DimOSBridge. As a result, publish_state() always returns at line 141 and never publishes odometry, GPS, or status data to the agent. This means the agent running alongside has no access to the drone's current position, heading, or status — effectively making it blind.

The fields _gps_pub and _status_pub are similarly unused.

[greptile-apps]

Recursive Timer chain can crash after stop()

The _check() closure accesses self._controller.body_id (line 275) without guarding against self._controller being None. After stop() is called, _running is set to False which prevents re-scheduling — but if a Timer fires between stop() and the GC clearing the controller reference, and _data is still not None, it will call self._controller.body_id which could raise AttributeError if the controller was cleaned up.

This same pattern applies to the land() _check closure at line 290. Consider adding a self._controller is not None guard:

Suggested change

	def _check() -> None:
	if self._data is not None:
	bid = self._controller.body_id
	if self._data.xpos[bid][2] >= altitude * 0.9:
	self._controller.set_velocity(0, 0, 0, 0)
	return
	if self._running:
	threading.Timer(0.2, _check).start()
	threading.Timer(0.5, _check).start()
	def _check() -> None:
	if self._controller is not None and self._data is not None:
	bid = self._controller.body_id
	if self._data.xpos[bid][2] >= altitude * 0.9:
	self._controller.set_velocity(0, 0, 0, 0)
	return
	if self._running:
	threading.Timer(0.2, _check).start()
	threading.Timer(0.5, _check).start()

[greptile-apps]

Silent exception swallowing in command listener

The broad except Exception on line 91 silently swallows all errors (including malformed JSON, missing keys, type errors in _handle_cmd) by sleeping and continuing. This will make debugging very difficult if a command arrives with unexpected structure. Consider logging the exception:

Suggested change

	except BlockingIOError:
	time.sleep(0.01)
	except Exception:
	time.sleep(0.01)
	except BlockingIOError:
	time.sleep(0.01)
	except Exception as e:
	print(f"[DimOS] Command error: {e}")
	time.sleep(0.01)

[greptile-apps]

hasattr check for timing state is fragile

Using hasattr(self, "_wall0") to lazily initialize timing state is brittle — if the sim loop is stopped and restarted (e.g., via stop() then start()), _wall0 and _sim0 will still be set from the previous run, causing an incorrect time offset. Consider initializing _wall0 and _sim0 in start() or at the top of _sim_loop() instead:

Suggested change

	# Real-time pacing
	sim_t = self._data.time
	wall_t = time.monotonic()
	if not hasattr(self, "_wall0"):
	self._wall0 = wall_t
	self._sim0 = sim_t
	dt = (self._wall0 + (sim_t - self._sim0)) - wall_t
	if dt > 0.001:
	time.sleep(dt)
	# Real-time pacing
	sim_t = self._data.time
	wall_t = time.monotonic()
	if step_count == 1:
	self._wall0 = wall_t
	self._sim0 = sim_t
	dt = (self._wall0 + (sim_t - self._sim0)) - wall_t
	if dt > 0.001:
	time.sleep(dt)