Function to bypass the compilation overrides on only particular refs

[benjikj-cera]

I'm trying to get more sophisticated with source selection during model development and would love some ideas on whether this is achievable via a Dataform macro/include.

Context: My compilation overrides replace the GCP project ID (swapping -prod for -dev) and append a schema suffix (_benjikj). This means all model ref() calls in my workspace point to dev tables.

The problem: When I'm developing a model mid-pipeline, I want per-ref control over whether the override applies. Specifically:

• Refs pointing to models I've been developing → use the dev table (compilation override applies as normal)
• Refs pointing to other sources I haven't touched → use the prod table (bypass the compilation override for those refs only)

This lets me test a downstream model against real prod data for untouched sources, while still pulling from my dev tables for anything I've actually been working on.

Proposed approach: I'm thinking of creating a ref_prod macro that wraps the standard ref() function but somehow suppresses the compilation overrides for that specific call — so I can swap in ref_prod only where I want prod data, and leave everything else as ref.

Is this achievable? Is there a better pattern for this kind of selective override suppression in Dataform?

[kolina]

I've shown an example here of a custom ref function

function customRef(ctx, name) {
    return ctx.ref("prod", ctx.schema(), name);
}

module.exports = { customRef };

The reason you're getting an error in your original question is that the ref function should point to a target that exists in a compiled DAG. So if you're using compilation overrides declaring dev sources in DAG, there won't be a prod source declared that ref could point to.

If you need to develop only a small amount of models (compared to the overall size of DAG), I'd suggest:

• Not use standard compilation overrides (so by default all models will use prod targets)
• Introduce custom compilation variables to override targets for a subset of developed models
• A regular ref should still correctly resolve dependencies
• Use action filters to only run your developed models (to make sure that any models depending on your developed models still pointing to prod are not run)

So your developed model can look like this

config {
    type: "table",
    project: dataform.projectConfig.vars.devProject,
    dataset: dataform.projectConfig.defaultSchema + "_" + dataform.projectConfig.vars.schemaSuffix
}

SELECT *
FROM ${ref(...)}