Skip to content

FIX: raise on characters AsciiSmugglerConverter cannot encode - #2540

Open
fei (feiiiiii5) wants to merge 1 commit into
microsoft:mainfrom
feiiiiii5:fix/ascii-smuggler-nonascii-fail-loud
Open

FIX: raise on characters AsciiSmugglerConverter cannot encode#2540
fei (feiiiiii5) wants to merge 1 commit into
microsoft:mainfrom
feiiiiii5:fix/ascii-smuggler-nonascii-fail-loud

Conversation

@feiiiiii5

Copy link
Copy Markdown
Contributor

Description

Addresses the data-loss bug reported in #2539.

Root cause. AsciiSmugglerConverter.encode_message maps each character in the ASCII printable range (0x20-0x7E) to a Unicode tag. Characters outside that range were collected into invalid_chars, logged via logger.error, and then dropped, but SmugglerConverter.convert_async still wrapped the truncated output in a success ConverterResult. So a non-ASCII objective was smuggled as a corrupted payload (café -> caf) or an empty one (你好世界 -> ""), while the run reported success. In a red-team scan that means an attack against a non-English objective can be scored "did not succeed" against a target that was never sent the objective.

Fix. Validate up front and fail loudly instead of silently dropping. encode_message now raises ValueError naming the offending characters and pointing to the lossless siblings. This matches the module's existing error contract (convert_async already raises ValueError for an unsupported input type, __init__ for an invalid action) and the sibling SneakyBitsSmugglerConverter / VariationSelectorSmugglerConverter, which round-trip arbitrary Unicode via UTF-8 bytes and never drop.

Behavior change. Encoding a prompt with any character outside 0x20-0x7E now raises instead of returning a success result with those characters removed. This only affects inputs the converter could never represent faithfully (it also covers control characters such as newline, dropped by the same path). Happy to carry a [BREAKING] tag on the title if you prefer.

Scope. I left the report's second, smaller point out of this PR: the decode_message length check that logs "No hidden Unicode Tag characters discovered." for an all-hidden string. That is a log-only inaccuracy with no effect on the returned value, so this stays scoped to the data-loss/false-success bug. I can send it as a follow-up or fold it in here, whichever you prefer.

Tests and Documentation

tests/unit/converter/test_ascii_smuggler_converter.py:

  • test_ascii_smuggler_encode_unrepresentable_chars_raise (parametrized over café, 你好世界, naïve résumé, an astral-plane emoji, and line1\nline2) asserts ValueError.
  • test_ascii_smuggler_encode_printable_boundaries_do_not_raise pins the inclusive 0x20/0x7E edges so the range check cannot drift by one.

Regression evidence:

  • Pre-fix on main (6d5b1a9): 5 failed, 7 passed - each new case DID NOT RAISE ValueError, captured log ERROR ...ascii_smuggler_converter.py:81 Invalid characters detected:.
  • Post-fix: 12 passed. All four token-smuggling converter test files: 38 passed.

Diff scope: 2 files, +38/-10 (ascii_smuggler_converter.py +17/-10, test +21). Formatted with the pinned ruff v0.16.4 (ruff format + ruff check -> all passed). No documentation change needed: the only doc sample that calls this converter (doc/code/converters/1_text_to_text_converters.py) uses the ASCII prompt "secret message".

encode_message collected characters outside the ASCII printable range
(0x20-0x7E), logged them at error level, and dropped them, while
convert_async still returned a success ConverterResult. A non-ASCII
objective was therefore smuggled as a corrupted or empty payload and the
run reported success. Raise ValueError naming the offending characters
and pointing to the lossless sibling converters instead.

Adds regression tests (pre-fix: DID NOT RAISE) covering the reported
non-ASCII cases, an astral-plane emoji, a control character, and the
inclusive 0x20/0x7E boundaries.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant