Intermittent clean "starvation" exit during early Linux boot with multithread sync policies (window closes only when the guest PL011 driver initializes)

[Jordanplus]

Summary

On a minimal aarch64 Linux platform (1× cpu_arm_cortexA53, arm_gicv3, SystemC Pl011 + char_backend_stdio, sync_policy = "multithread-unconstrained"), a significant fraction of runs (~1 in 4 on our host) exits cleanly at ~0.11–0.12 s of simulated time during early kernel boot, printing e.g. Simulation Time: 0.119137 SC_SEC and terminating as if the simulation had ended normally.

The mechanism is deterministic; the trigger is host-thread-timing dependent, hence probabilistic. All code cited below was verified identical between v7.3.0 (where we observed the behavior) and current main (c00e7f7).

We understand that exiting on "starvation" (no timed events + no attached suspending channels) is documented, intended behavior — see the comment in cpu.h#L706-L716 ("... allowing normal starvation exit"). This report is about the boot-time gap between that semantic and the moment the platform first gains a persistent suspending channel.

Mechanism

1. Under multithread-* sync policies, the only thing keeping the SystemC kernel alive while a vCPU computes is the vCPU quantum keeper's m_tick suspending channel:

   • m_tick declaration:

     qbox/systemc-components/common/include/qkmultithread.h

     Line 34 in c00e7f7

     async_event m_tick;

   • attached in start():

     qbox/systemc-components/common/src/libgssync/qkmultithread.cc

     Lines 101 to 111 in c00e7f7

     	void tlm_quantumkeeper_multithread::start(std::function<void()> job)
     	{
     	SCP_TRACE(())("Start");
     	status = RUNNING;
     	m_tick.async_attach_suspending();
     	m_tick.notify(sc_core::SC_ZERO_TIME);
     	if (job) {
     	std::thread t(job);
     	m_worker_thread = std::move(t);
     	}
     	}

2. When a vCPU executes WFI with no pending work, wait_for_work() stops its QK (cpu.h#L347-L369, m_qk->stop() at L350). stop() notifies m_tick with SC_ZERO_TIME (qkmultithread.cc#L113-L129, L120) — this delta notification also supersedes any still-pending timed tick notification on the same event — and the timehandler (on the SystemC thread) then detaches the suspending channel (qkmultithread.cc#L27-L46, detach at L44). Net effect: an idle vCPU contributes neither a suspending channel nor a pending timed event.

3. On such a platform the only other candidate suspending channel is the serial path, and it is armed by the guest: the stdio backend's biflow m_send_event becomes attach-suspending only once the UART side calls can_receive_set(<nonzero>):

   • biflow control handling (attach at L121 / detach at L123):

     qbox/systemc-components/common/include/ports/biflow-socket.h

     Lines 107 to 124 in c00e7f7

     	void initiator_ctrl(tlm::tlm_generic_payload& txn, sc_core::sc_time& t)
     	{
     	std::lock_guard<std::mutex> guard(m_mutex);
     	sc_assert(txn.get_data_length() == sizeof(ctrl));
     	ctrl* c = (ctrl*)(txn.get_data_ptr());
     	switch (c->cmd) {
     	case ctrl::ABSOLUTE_VALUE:
     	m_can_send = c->can_send;
     	m_infinite = false;
     	SCP_TRACE(())("Can send {}", m_can_send);
     	// If the other side signals a specific number, then it must be waiting on interrupts
     	// etc to drive this, so lets interpret that as a 'signal' that we should keep the
     	// suspending flag. If this is set to 0, then the other side stopped listening
     	if (m_can_send)
     	m_send_event.async_attach_suspending();
     	else
     	m_send_event.async_detach_suspending();
     	break;

   • can_receive_set() API note:

     qbox/systemc-components/common/include/ports/biflow-socket.h

     Lines 243 to 259 in c00e7f7

     	/**
     	* @brief can_receive_set
     	*
     	* @param i number of items that can now be received.
     	*
     	* Setting this to zero will have the side effect of async_detach_suspending the other ends
     	* async_event. Setting this to NON-zero will have the side effect of async_attach_suspending
     	* the other ends async_event.
     	*/
     	void can_receive_set(int i)
     	{
     	ctrl c;
     	c.cmd = ctrl::ABSOLUTE_VALUE;
     	c.can_send = i;
     	send_ctrl(c);
     	}
     	/**

   • The SystemC PL011 calls backend_socket.can_receive_set(16) only from pl011_set_read_trigger() (uart-pl011.h#L264-L280, L279), which runs when the guest writes UARTLCR_H (#L309-L317) or UARTIFLS (#L322-L325) — i.e. when the Linux pl011 driver initializes.

4. Therefore, between simulation start and pl011 driver init there is a window in which a single "all idle" instant — the CPU in WFI, its QK detached, the event queue drained — satisfies the starvation condition and the simulation ends cleanly. After the driver probes the UART, can_receive_set(16) keeps the backend's suspending channel attached permanently and the window closes for good.

Whether the all-idle instant actually occurs inside the window depends on host scheduling, which is why the failure is intermittent.

Reproduction

1. Assemble a minimal aarch64 Linux platform from in-tree components (structure as in docs/libqbox.md and docs/backends.md#L111-L127): 1× cpu_arm_cortexA53, arm_gicv3, Pl011 (dylib_path = "uart-pl011") bound to char_backend_stdio, gs_memory, a router, and a QemuInstance with tcg_mode = "MULTI", sync_policy = "multithread-unconstrained". Do not instantiate keep_alive.
2. Boot a standard arm64 Linux Image + DTB.
3. Run the boot 10–20 times. On our host, 3 of 12 runs exited cleanly at 0.113–0.124 s simulated time (before the kernel serial console comes up), printing the normal end-of-simulation banner. After adding a keep_alive instance, 12 of 12 runs booted to completion.

The in-tree platforms/ubuntu aarch64 configuration is latently affected: it uses the same multithread-unconstrained policy (conf_aarch64.lua#L77) and the same Pl011 + char_backend_stdio pair (#L118-L129) with no keep_alive; its 8 CPUs (#L48) merely make the all-idle instant rarer, they do not eliminate the window.

Suggestions (either would resolve this)

1. Document that Linux-boot (or generally interrupt-driven-idle) platforms under multithread-* policies must instantiate the in-tree keep_alive module (keep_alive.h#L20-L36 — it simply keeps an async_event attach-suspending). docs/base-components.md#L363-L384 already describes the component generically ("prevents the SystemC simulation from ending prematurely", e.g. under "QEMU-driven execution") and would be the natural place for this note, but nothing there connects it to the starvation-exit window above or states that Linux-boot multithread-* platforms need it — and today only the cortex-m55-remote platform instantiates it (conf.lua#L47-L48; no other in-tree .lua config does).
2. Or offer an opt-in liveness parameter on QemuInstance that keeps a suspending channel attached as long as any vCPU exists (attached at start_of_simulation, released when the CPUs finish), so platforms can choose "run until the guest shuts down" semantics without a separate module, while the default starvation semantics stay unchanged for existing tests.

Workaround

Add a keep_alive instance to the platform. Note for others applying this: guest-initiated shutdown (poweroff / PSCI SYSTEM_OFF) still terminates the process, because the libqemu iothread calls exit(status) when the QEMU main loop returns — see libqemu/libqemu.c#L90-L114 (exit(status) at L113) on the libqemu-v11.0-v0.10 tag pinned by QBox (package-lock.cmake#L29-L32) — so run scripts keyed on process exit keep working. However, sc_main no longer returns: end_of_simulation callbacks and the final "Simulation Time:" banner are skipped.