fix(html-to-markdown): preserve nested list structure in html → markdown - #247
Merged
Merged
Conversation
The non-greedy <ul>(.*?)</ul> / <ol>(.*?)</ol> regexes paired an outer open tag with an inner close tag, flattening nested items and dropping the marker of the next sibling (data loss). Replace only the two list passes with a balanced-tag scanner that mirrors StorageWalker's handleList/renderListItemBody output and indents nested lines with the shared LIST_INDENT sentinel. Tags are paired in a single linear pass, nesting deeper than 256 levels is flattened as before, and decoded numeric entities are sentinel-escaped. Flat-list output and all other html-to-markdown behavior are unchanged. Fixes #243
8 of 15 tasks
Text after a nested list is emitted as a LIST_INDENT continuation line. A flattened <pre> there (``` x ```) was taken by splitOnFences for a sentinel-indented fence opener and paired with the next top-level fence, sending that code body through whitespace cleanup. Split item text on fences and keep fence segments as line-preserving blocks, joining a leading fence to the marker with LIST_INDENT (as StorageWalker does) only when it is a real fence. <pre> inside any list item now renders as a fenced block instead of being flattened. Also drop self-closing <ul/> / <ol/> before tag normalisation so they cannot steal the enclosing list's close tag, and update markdown-cleanup comments that described LIST_INDENT as StorageWalker-only.
…nested-lists # Conflicts: # lib/markdown-cleanup.js # tests/html-to-markdown.test.js
|
🎉 This PR is included in version 2.25.5 🎉 The release is available on: Your semantic-release bot 📦🚀 |
This was referenced Sep 30, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request Template
Description
Fixes #243.
convert --input-format html --output-format markdownuses the regex-basedlib/html-to-markdown.js. Its non-greedy/<ul>(.*?)<\/ul>/gsand/<ol>(.*?)<\/ol>/gspatterns paired the outer opening tag with the inner closing tag. That flattened nested items and dropped the marker of the next sibling:Options considered
StorageWalker<pre><code class="language-x">into inline code and drops the language.tests/convert.test.jsexplicitly guards against this, and inline/entity handling would change as well.What changed (
lib/html-to-markdown.js)tokenizeLists,buildListandrenderList. Tags are paired in a single linear pass.<li>implicitly closes an open sibling item.StorageWalker#handleList/renderListItemBodyfrom fix(storage-walker): preserve nested list structure in storage → markdown (#238) #239.<p>, which stay on one line (- a b), whereas the walker splits them into separate paragraphs.<pre>inside an item stays a fenced block with its lines intact. A fence that leads the item is joined to the marker withLIST_INDENT, as in the walker, sosplitOnFencesskips its body. That join is used only for real fences: literal text starting with ``` keeps a plain space, so it can't pair with a later fence.storage → markdown.LIST_INDENTsentinel, so the whitespace cleanup chain cannot collapse them. Input sentinels are escaped, and so are decoded numeric entities ().<ul/>/<ol/>are dropped before tag normalization, so they can't steal the enclosing list's close tag.StorageWalker'sDEFAULT_MAX_DEPTH) is flattened the way the old code did, so rendering cannot overflow the stack.Behavior vs. v2.25.3: output for flat lists of inline content and for all non-list constructs is byte-identical.
Intentional change:
<pre>inside a list item now renders as a real fenced block instead of being flattened onto the marker line. This also fixes a regression found in review, where a<pre>following a nested list was mis-paired with a later top-level fence and that code body lost its whitespace:<ul><li><pre><code>a b</code></pre></li></ul>- ``` a b ```- ```\n a b\n ```<ul><li><pre><code class="language-js">a\nb</code></pre></li></ul>- ```js a b ```- ```js\n a\n b\n ```Malformed HTML: these differ, and in most of them the old code lost items:
<ul><li>a</li><li>b</ul>- a- a\n- b<ul><li>a<li>b</ul>- a\n- b<ol><li>a</li><ol><li>b</li></ol></ol>1. a\n2. b1. a\n 1. b<ul><li>a</li></ol><ol><li>b</li></ul>- a\n- b- a\n\n1. b<ul><li>a<ul><li>inner</li></ul>(outer never closed)- a innera\n- innerKnown follow-ups (out of scope here):
(not a `<pre>`) can still pair with a later top-level fence. Example: `<ul><li>a<ul><li>b</li></ul>followed by astart/typeattributes on<ol>.<strong>wrapping a nested list.Unclosed lists are also much faster: 20k unclosed
<ul><li>x</li>went from 514 ms to 12 ms.Type of Change
Testing
npx jest: 1329 tests pass (after merging main with #249).npx eslint lib tests: clean.tests/html-to-markdown.test.js: new exact-output tests under "nested lists (#243)" cover:ulandol, mixed nesting 3 levels deep, and a multi-digit marker indent</li>, unclosed and mismatched closes<pre>inside a nested itemtests/convert.test.js: a CLI end-to-end test.Checklist
Screenshots (if applicable)
N/A
Additional Context
Follow-up to #238 / #239.
renderListItemBody()fixed nested lists in the storage path. This PR brings the html path to the same output without touching its other passes.