Experimental JS API based on tracking values by jgonet · Pull Request #2013 · atomvm/AtomVM

jgonet · 2025-12-01T15:42:11Z

This PR upstreams API we're using in Popcorn.
It doesn't have tests or extended docs. I'd gladly take some instructions about testing Wasm. Docs will be added after first round of review – I need to know if approach is sane at all.

Design notes

We needed some API to track values in JS with their lifetime in the VM. This was achieved by using new type of resource – TrackedValue. Popcorn also wants to customize JS behavior without maintaining custom Emscripten hooks in .c files – we extracted them to functions in atomvm.pre.js. You can see our implementation in Popcorn repo.

We also need JS to have access to DOM. We hardcoded thread executing JS to be main thread (or in our case, iframe thread).
We were thinking about relaxing that constraint but it'd need js_tracked_eval to forward script to it's destination JS context (e.g. postMessage() to iframe).

js_get_tracked_objects works by batching TrackedObjects and their status (missing, bad type, etc). This allows to reduce communication overhead between JS and Wasm.

We also needed to use ES modules instead of current, IIFE format. It makes it easier to load in iframes and to use in modern JS projects.

These changes are made under both the "Apache 2.0" and the "GNU Lesser General
Public License 2.1 or later" license terms (dual license).

SPDX-License-Identifier: Apache-2.0 OR LGPL-2.1-or-later

Signed-off-by: Jakub Gonet <jakub.gonet@swmansion.com>

It's a field in emscripten platform struct. It could be a callback entirely in JS but we use threads. Threads are emulated via webworkers which have their own contexts and variables aren't shared with each other. This means that we would lose uniqueness of the keys. This commit also changes output format from file loaded and ran at <script src=...> to ES6 classes which have greater control over initialization of Wasm. Signed-off-by: Jakub Gonet <jakub.gonet@swmansion.com>

Signed-off-by: Jakub Gonet <jakub.gonet@swmansion.com>

jgonet · 2025-12-01T15:42:31Z

cc @pguyot

bettio

I added just some minor comments.
I'm not the emscripten expert here, however it would be good (and useful) having some documentation in emscripten module (and where it makes sense) about how this is supposed to be used. Some additional documentation would be super appreciated during review.

bettio · 2025-12-23T12:35:22Z

src/platforms/emscripten/src/lib/platform_nifs.c

+        term result = term_invalid_term();
+        term refs = term_nil();
+        if (UNLIKELY(memory_ensure_free_opt(target_ctx, TUPLE_SIZE(2) + LIST_SIZE(keys_n, TERM_BOXED_REFC_BINARY_SIZE), MEMORY_CAN_SHRINK) != MEMORY_GC_OK)) {
+            // TODO: how to raise?


Is this TODO still valid?

It is. I pretty much copied this code from Emscripten function running JS script async (nif_emscripten_run_script). As far, as I understand it, we set trap signal and trigger it by returning term_invalid_term() which makes process wait for the signal to arrive.

I'm not sure how to do the signal sending part and raise at the same time. Should I just set result = term_invalid_term and override regs?

bettio · 2025-12-23T12:42:34Z

src/platforms/emscripten/src/lib/platform_nifs.c

+        free(keys);
+        mailbox_send_term_signal(target_ctx, TrapAnswerSignal, result);
+        globalcontext_get_process_unlock(global, target_ctx);
+    } else {


style: you can either remove this } else { and just free() at the end of function, or even do an early return when target_ctx == NULL. Maybe the second option is more readable.

I'm fine with either. Should we also change run_script to do it the same?

bettio · 2025-12-23T12:45:08Z

src/platforms/emscripten/src/lib/platform_nifs.c

+});
+// clang-format on
+
+static void do_get_tracked_objects(uint32_t *ref_keys, size_t keys_n, int32_t sync_caller_pid, GlobalContext *global)


Right now we are using uint32_t for process ids. Also we are using the extended process_id name instead of pid when talking about the id itself (not encoded as a term having type pid).

bettio · 2025-12-23T12:47:01Z

src/platforms/emscripten/src/lib/platform_nifs.c

+});
+// clang-format on
+
+static void do_run_script_tracked(const char *script, int32_t sync_caller_pid, GlobalContext *global)


Right now we are using uint32_t for process ids. Also we are using the extended process_id name instead of pid when talking about the id itself (not encoded as a term having type pid).

bettio · 2025-12-23T13:45:26Z

src/platforms/emscripten/src/CMakeLists.txt

-target_link_options(AtomVM PRIVATE -sEXPORTED_RUNTIME_METHODS=ccall -sUSE_ZLIB=1 -O3 -pthread -sFETCH -lwebsocket.js --pre-js ${CMAKE_CURRENT_SOURCE_DIR}/atomvm.pre.js)
-
+if (AVM_USE_WASM_MJS)
+    target_link_options(AtomVM PRIVATE -sEXPORTED_RUNTIME_METHODS=ccall,cwrap,stringToNewUTF8 -sEMULATE_FUNCTION_POINTER_CASTS=1 -sEXPORTED_FUNCTIONS=_malloc,_cast,_call,_next_tracked_object_key,_main -sEXPORT_ES6=1 -sUSE_ZLIB=1 -O3 -pthread -sFETCH -lwebsocket.js --pre-js ${CMAKE_CURRENT_SOURCE_DIR}/atomvm.pre.js)


-O3 if a pretty aggressive optimization option. Do we really want to hardcode it?

Agreed. I vaguely recall that CMake has some standard compiler/linker env vars so we could use them. Either way, choice between -O3 and -Os would be nice.

I just retained current state which hardcodes -O3.

bettio · 2025-12-23T13:50:34Z

src/platforms/emscripten/src/lib/platform_nifs.c

+{
+    static const uint8_t OK = 0;
+    static const uint8_t BAD_KEY = 1;
+    static const uint8_t NOT_STRING = 2;


Shouldn't we define an enum here?

I think I did it that way out of reflex when writing JS code above. I'll change it.

bettio · 2025-12-23T13:59:46Z

src/platforms/emscripten/src/lib/platform_nifs.c

        if (strcmp("run_script/2", nifname) == 0) {
            return &emscripten_run_script_nif;
        }
+        if (strcmp("run_script_tracked/1", nifname) == 0) {


They should be added to emscripten module: libs/eavmlib/src/emscripten.erl
typespecs and documentation about how they should be used is appreciated.

jgonet added 5 commits December 1, 2025 14:24

wasm: adapt cmakelists

099f447

Signed-off-by: Jakub Gonet <jakub.gonet@swmansion.com>

wasm: add JS scaffolding for tracked values

9859c62

Signed-off-by: Jakub Gonet <jakub.gonet@swmansion.com>

wasm: add js_tracked_eval

e2a76b7

Signed-off-by: Jakub Gonet <jakub.gonet@swmansion.com>

wasm: add get_tracked

4c1a5d8

Signed-off-by: Jakub Gonet <jakub.gonet@swmansion.com>

bettio reviewed Dec 23, 2025

View reviewed changes

bettio requested a review from pguyot December 23, 2025 14:10

Conversation

jgonet commented Dec 1, 2025

Design notes

Uh oh!

jgonet commented Dec 1, 2025

Uh oh!

bettio left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

bettio Dec 23, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

bettio Dec 23, 2025 •

edited

Loading