Proposal: Optional strict semantic boundaries for structured queries

[KentonParton]

An MCP tool normally presents an explicit capability boundary: the agent may use the resources, fields, and operations described by the tool schema, but it cannot reach beyond that interface.

A BSL model describes a similar contract through its dimensions, measures, and relationships. However, BSL does not currently enforce that contract as a strict query boundary. A structured query can reference columns that fall outside the semantic model when those columns still exist on the underlying Ibis relation.

This means a model may appear to expose only a defined set of semantic members while queries can still access undeclared implementation columns. For MCP and other agent-facing use cases, it would be valuable for BSL to optionally enforce the semantic model as the complete queryable boundary.

Minimal reproduction

import ibis
from ibis import _
from boring_semantic_layer import to_semantic_table


source = ibis.memtable(
    {
        "status": ["open", "closed", "open"],
        "private_amount": [100, 200, 300],
    }
)

opportunities = (
    to_semantic_table(source, name="opportunities")
    .with_dimensions(
        status=_.status,
    )
    .with_measures(
        opportunity_count=_.count(),
        pipeline_amount=_.private_amount.sum(),
    )
)

print("Declared dimensions:", sorted(opportunities.get_dimensions()))
print("Declared measures:", sorted(opportunities.get_measures()))

print(
    opportunities.query(
        dimensions=["private_amount"],
        measures=["opportunity_count"],
    ).execute()
)

print(
    opportunities.query(
        dimensions=["status"],
        measures=["opportunity_count"],
        filters=[
            {
                "field": "private_amount",
                "operator": ">=",
                "value": 200,
            }
        ],
    ).execute()
)

Output on boring-semantic-layer 0.3.14 from commit 1c05d19c95262856541e875c40ca4f2c13f21608:

Declared dimensions: ['status']
Declared measures: ['opportunity_count', 'pipeline_amount']

   private_amount  opportunity_count
0             200                  1
1             300                  1
2             100                  1

   status  opportunity_count
0    open                  1
1  closed                  1

Expected: both direct references to private_amount should be rejected because it is not a declared dimension or measure.

[KentonParton]

@hachej Thanks for putting together #277 — I really appreciate you taking this on.

Is there anything I can do to help get this over the line and merged?

Thanks again

[hachej]

Hi @KentonParton we are almost there :) Could you test quickly the current implementation ? Thanks a lot, we should be able to merge this on Wednesday

[KentonParton]

Yes, sure. Will give it a try in the AM.

[KentonParton]

Thanks @hachej — I tested the current PR branch and the strict boundary behavior works as expected for the Python query API.

Verified:

• undeclared physical columns are rejected as dimensions with strict_semantic_boundaries=True
• undeclared physical columns are rejected in structured dict filters
• declared measures can still use private physical columns internally
• model-prefixed declared fields still work
• default behavior remains permissive when strict_semantic_boundaries is not enabled

I also checked order_by, nested filters, and having; those are rejected correctly when they reference undeclared fields. The focused regression tests pass, and the adjacent query/filter tests passed locally as well.

The only thing I noticed is scope-related: the MCP and HTTP query entrypoints do not currently expose/pass strict_semantic_boundaries=True. Would be great to include strict_semantic_boundaries=True in the MCP and HTTP query entrypoints too.

[KentonParton]

I dug into the MCP/HTTP surfaces a bit more, and I think compare_periods should be included in the same scope.

The reason is that compare_periods is also an agent-facing structured query surface: it accepts dimensions, measures, filters, time_dimension, and order_by. If MCP/HTTP expose compare_periods without strict semantic boundaries, an agent could still influence the result set using an undeclared physical column in a filter, even if regular query_model is locked down.

Suggested update:

• pass strict_semantic_boundaries=True from the MCP query_model tool
• pass strict_semantic_boundaries=True from the HTTP /query endpoint
• add strict_semantic_boundaries to the core compare_periods API
• pass it through to the internal query(...) calls inside compare_periods
• pass strict_semantic_boundaries=True from MCP compare_periods
• pass strict_semantic_boundaries=True from HTTP /compare-periods

That would make the MCP/HTTP structured surfaces line up with the original capability-boundary goal.

[KentonParton]

I did an adversarial pass of the recent updates across Python, MCP, HTTP, query_model, compare_periods, and dimension-value lookup.

The structured MCP/HTTP surfaces look good now: raw dimensions, measures, filters, nested filters, order_by, compare_periods time_dimension, and dimension-value lookup are rejected.

I also tried code-like payloads inside structured field, operator, and value entries. Those were either rejected or treated as literal values; I did not observe code execution through the structured dict filter interface.

I did find one remaining bypass class in non-structured Python/query-string surfaces. With strict_semantic_boundaries=True, string and callable filters can still reference raw physical columns:

filters=["_.private_amount >= 200"]
filters=[lambda t: t.private_amount >= 200]
Filter(filter="_.private_amount >= 200")
Filter(filter=lambda t: t.private_amount >= 200)

The same applies through compare_periods.

There is also a more serious version of this issue: string-eval surfaces can execute Python code if exposed to untrusted input. I was able to execute a harmless sentinel-file payload through a string filter while still returning a valid query result, e.g. a string filter shaped like:

payload = (
    "ibis.__builtins__['__import__']('pathlib')"
    ".Path('/private/tmp/bsl_probe_sentinel').write_text('executed')"
)

opportunities.query(
    dimensions=["status"],
    measures=["opportunity_count"],
    filters=[f"{payload} and (_.status == 'open')"],
    strict_semantic_boundaries=True,
).execute()

The older BSLTools/OpenAI-style query_model path also evaluates an arbitrary query string, so an agent using that surface can both bypass semantic boundaries, e.g. call:

opportunities.query(dimensions=["private_amount"], measures=["opportunity_count"])

and potentially execute Python code before returning a normal BSL query result.

For a strict security/capability boundary, I think strict_semantic_boundaries=True should reject opaque filters that cannot be inspected safely: string filters, callable filters, and Filter objects wrapping strings/callables.

The structured dict filter interface still seems like the right safe filtering mechanism for agents.