[CalvinTjoaquinn]

Fixes #356

html_escape(s, true) writes one numeric character reference per UTF-8 byte. A numeric reference names a Unicode code point, so anything outside ASCII comes out as a reference sequence that decodes to mojibake instead of the original text.

Measured

html_escape("こんにちは", true)
before: こんにちは
after:  こんにちは

html_unescape(html_escape("こんにちは", true))
before: "ã\^A“ã‚“ã\^A«ã\^A¡ã\^A¯"
after:  "こんにちは"

html_escape("🚀", true)
before: 🚀
after:  🚀

こ is U+3053, so こ is the reference that names it. ã names U+00E3, which is ã, and that is exactly what a browser renders.

Why

r := rune(escaped[i])
fmt.Fprintf(&result, "&#%d;", int(r))
i++

rune(escaped[i]) converts a single byte, so こ (E3 81 93) became three references. The fix decodes a rune with utf8.DecodeRuneInString and advances by its width.

A byte that is not valid UTF-8 keeps its old spelling, since there is no code point to name it with, so binary input behaves as it did before.

Nothing else changes

The existing assertion is ASCII only and is byte-identical before and after:

html_escape("<body>test</body>", true)
-> &lt;&#98;&#111;&#100;&#121;&gt;&#116;&#101;&#115;&#116;&lt;&sol;&#98;&#111;&#100;&#121;&gt;

Characters the escaping pass already has a named entity for never reach this loop. é becomes é and is copied across untouched, which the new test pins so a later change does not quietly start renumbering them.

Verification

Reverting util.go and keeping the new test:

--- PASS  ascii is one reference per character
--- FAIL  cjk
--- FAIL  outside the basic multilingual plane
--- FAIL  multi-byte characters stay separate
--- PASS  named entity from the escaping pass is left alone

Each case also asserts html_unescape returns the original string, which is the property that was broken.

With the change: go test -race ./... passes and golangci-lint run ./... reports 0 issues.

Separate from #340 and #341, which touch url_encode in dsl.go. No overlap.