[1/3] FPI

[greenhat]

Ref #697

Adds end-to-end Foreign Procedure Invocation support for SDK account components and note scripts. The new #[foreign_account(package_name1, package_name2)] SDK macro now discover Miden account dependencies from package metadata, synthesize caller-only FPI imports, and generate methods for the struct that let callers invoke dependency methods through execute_foreign_procedure with normal Rust signatures.

• During binding generation, the macro loads the selected WIT world and injects a synthetic fpi-<method> import for each freestanding method exported by a Miden dependency interface. Each synthetic import prepends the foreign account id prefix, account id suffix, and procedure root word to the original method parameters while preserving the original result type. wit-bindgen then renders these imports as Rust fpi_<method> free functions.
• After wit-bindgen emits bindings, the macro scans dependency import modules for generated fpi_* functions, loads the dependency .masp package, resolves exported procedure roots, and appends methods to the defined struct. The public wrapper methods remove the internal prefix/suffix/root parameters from the user-facing signature and call the generated fpi_* function with self.0.prefix, self.0.suffix, and the resolved root.
• The Wasm frontend recognizes imports whose WIT function name starts with fpi- and routes them through dedicated FPI lowering instead of generic component import lowering. For direct canonical ABI calls, it rewrites the inner import call into an HIR exec of miden::protocol::tx::execute_foreign_procedure, reordering the account id to the protocol ABI order, expanding the procedure root word, forwarding flattened procedure inputs, validating the 16-input/16-result limits, and storing multi-felt results through the canonical output pointer when needed.
  For FPI calls whose flattened parameters exceed the direct canonical ABI limit, the canonical ABI passes a single argument tuple pointer, and the generated wrapper function reloads every flattened value from that tuple in HIR (sign-extending narrow signed integers), so the indirect shape never leaves the frontend. From there, every FPI call — direct, indirect, and raw — lowers to the same dedicated hir.exec_fpi op: its operands are the up-to-16 flattened procedure input felts, while the 6-felt executor prefix (account id suffix and prefix, already reordered, plus the procedure root word) is stored into felt function locals referenced by the op. MASM codegen then pads the scheduled inputs with zeros to the fixed 16-felt executor input width, loads the prefix locals on top of the stack, and invokes the real execute_foreign_procedure — no compiler-internal marker callee, no layout attributes, and no stack shuffling beyond the addressable 16-element window.

The note to account example:

...
#[foreign_account(mixed_scalar_struct_account)]
struct CounterContract;
...
#[note]
impl CounterCaller {
    /// Checks that a mixed scalar record crosses the FPI boundary in both directions.
    #[note_script]
    pub fn run(self, _arg: Word) {
        let count_acc = CounterContract::from_account(self.counter_account_id);
        let result = count_acc.echo_mixed_scalar_record(MixedScalarRecord {
            first_u64: FIRST_U64,
            second_u64: SECOND_U64,
            felt_value: felt!(77),
            u32_value: U32_VALUE,
            u8_value: U8_VALUE,
        });

See tests/integration-network/src/mockchain/fpi/note/mixed_scalar_struct.rs for the complete example.

The account to account example:

#[foreign_account(account_to_account_callee_account)]
struct CalleeAccount;

/// Account component which forwards reads to another account through FPI.
#[component]
struct CallerAccount;

#[component]
impl CallerAccount {
    /// Reads a counter value from the provided foreign account.
    pub fn read_foreign_count(&self, callee_account_id: AccountId) -> Felt {
        let callee = CalleeAccount::from_account(callee_account_id);
        let key = Word::new([felt!(13), felt!(21), felt!(34), felt!(55)]);
        callee.get_count(key)
    }
}

See tests/integration-network/src/mockchain/fpi/account/account_to_account.rs for the complete example.

The note to account example with multiple packages:

#[foreign_account(multiple_packages_first_account, multiple_packages_second_account)]
struct ForeignCounters;

/// Note script input containing the foreign account id.
#[note]
struct CounterCaller {
    /// Account id with both counter components deployed.
    foreign_account_id: AccountId,
}

#[note]
impl CounterCaller {
    /// Checks that a multi-package foreign account binding exposes both component methods.
    #[note_script]
    pub fn run(self, _arg: Word) {
        let counters = ForeignCounters::from_account(self.foreign_account_id);

        let first = counters.get_first_count();
        let second = counters.get_second_count();
...

This PR also adds raw bindings for the kernel execute_foreign_procedure procedure so that the MASM account methods can be called via FPI:

...
let outputs = tx::execute_foreign_procedure(
    self.foreign_account_id,
    procedure_root,
    inputs,
);
...

[greenhat]

@bitwalker @bobbinth I finished the FPI implementation with the #[foreign_account(package_name1, package_name2)] attribute macro we discussed in #697 (comment). Check out the PR description and tests for examples.

EDIT: We support account-to-account, note-to-account, and direct execute_foreign_procedure calls (for MASM accounts).

[greenhat]

Following our discussion to go with the 2.1 #[account(...)] macro for both current and foreign accounts, I renamed the #[foreign_account(...)] macro to #[account(...)] in 9966e55

[bitwalker]

I'm still working my way through this, but I think some additional documentation in the codegen backend would be helpful here, as there are a few aspects that aren't super clear to me that I suspect documentation would fix.

[bitwalker]

Could the module doc here elaborate a bit more on how FPI is handled during lowering? In particular, the aspects related to ABI handling, but I would also document any assumptions/expectations on code that triggers the FPI handling.

[greenhat]

Done in d9c4587

[bitwalker]

Maybe it would be simpler if we arranged to store the first 6 operands (account id, procedure root, FPI context) in locals, and then load them on to the operand stack manually in the lowering of an hir.exec that targets execute_foreign_procedure after padding the stack out to 16 elements, thereby getting all 22 operands on the operand stack with minimal fuss.

With only a single scratch local, it takes a lot more shuffling to get things into place.

This also seems to me to be a good fit for a special instruction specifically for FPI, that largely looks like hir.exec, but rather than the first 6 operands (3 values) being SSA values, they would be references to 3 local variables that are stored to prior to the FPI instruction. Lowering for the new instruction would then be able to guarantee that up to 16 operands would be on the operand stack on entry to the lowering of that op, and the lowering could manually load the values of the 3 local variables on top of the operand stack in the correct order. The only special handling that would be needed in the lowering code would be the padding aspect.

What do you think? It's a bit of an adjustment, but I think it would make lowering more straightforward, more efficient, and a bit easier to reason about (not to mention it would avoid complicating the lowering for hir.exec.

[greenhat]

Great idea! Thank you! Implemented in the last two commits (6119c12 and 39dafb8)

[bitwalker]

I can't help but feel it would be better if we could somehow use the "real" signature of the foreign procedure somehow, probably by storing the type signature in an attribute and making the operands/results AnyType - but I see that more as a future improvement - this will do for now I think.

[greenhat]

I agree. I made #1176.