Bookmark text of static reference is lost after generating with the option to update fields is set.

[ylussaud]

See this discussion for more details.

Project example:
Bookmark_issue.zip

[nheuermann]

Hi,

I investigated this with Copilot using the provided example files and a second internal example (very helpful for direct comparison on XML level). We had users reporting similar issues, but we could never really pinpoint the issue.
To my understanding, the issue is that M2Doc does not correctly handle bookmarks (the targets of the references) - they are either removed (if spanning multiple paragraphs), reduced to size 0 and thus references cannot show any text (if bookmark start/end are within the same paragraph) or retained (if the paragraph is raw copied - I couldn't fully understand how this could be identified on user level). So seems to be a quite general problem for bookmarks.

The current M2Doc approach to re-generate most paragraphs and only raw copy certain static paragraphs may also damage other less visible attributes of the docx file. Longer term, it might be an idea to change to an approach where more parts of the documents are raw-copied and only the really necessary ones are re-generated.

I've just investigated, without providing a direct fix. Not being a Java developer, I could try to orchestrate AI for some fix, but my current assumption is that you'd prefer to do a more complex fix as for this issue on your own - right, Yvan?

Copilot summary of findings:
The issue is caused by M2Doc reconstructing template content at the OOXML paragraph level. Ordinary Word bookmarks are not represented as a paired/ranged structure by M2Doc. During generation, paragraph properties are copied, runs are removed and recreated, but w:bookmarkStart and w:bookmarkEnd are not kept in their original position relative to the runs.

Bookmarks contained within one reconstructed paragraph remain by name, but their start and end markers are emitted together before the recreated content, making the bookmark zero-length.
Bookmarks spanning multiple paragraphs or larger structures are not tracked across those containers and can be removed entirely.
A REF field may retain its instruction and cached result, but its target bookmark can already be empty. When fields are marked dirty and Word recalculates them, the reference text is consequently lost.
In another example, the REF field’s separator and cached result were also absent from the generated output. This appears to be a related field-reconstruction effect, rather than an AQL evaluation issue.
The affected code paths are primarily M2DocEvaluator’s run-by-run paragraph reconstruction (createNewParagraph, insertRun, and insertFragment) and RawCopier’s partial paragraph copying (copyParagraphFragment). The latter also has no bookmark state handling across paragraph boundaries. Tables are parsed and regenerated structurally as M2Doc Table objects even when they contain no M2Doc commands, so captions and surrounding bookmark ranges are also exposed to this behavior.

This appears to be a regression associated with the low-level XML copier introduced in commit fa49cf1 (Fixed UserContent copy to use low level xml structure), and particularly the follow-up commit c14bd83 (Removed RawCopier.copyFromRun()), which removed special handling that preserved markup such as bookmark starts when copying from a run. The current behavior is therefore best understood as a regression in preservation of OOXML child ordering and bookmark range state, not as intentional handling of ordinary Word bookmarks.

More details in this attached file (could be used for continuing the work with AI): M2Doc_Bookmark_Investigation_Detailed.txt

[ylussaud]

Thank you for your investigation. That's very interesting and referenced commits seem accurate. Maybe this can be fixed in the RawCopier I can't really remember why I removed this at the time. But maybe a better idea would be to identify bookmark start and end in the AST produced by the parser to make sure they are preserved.

But some limitations will exists. For instance if the start of a bookmark is not in the same block of instruction has the end of the bookmark, it might be generated several times (in a for loop) or omitted (in a if statement)... But I think we can live with that.

PS: I still need to read the all report.

[nheuermann]

Having an internal list of bookmark start/ends and recreating these would be the better idea, I agree.
That repeated blocks are not allowed to only have start/end included is clear - it would be great if M2Doc generation could give a meaningful warning message to the user, so that one can correct the Word bookmark by hand (hopefully having enough information from the warning message).

[pgesposito]

Hi, with the support of ChatGPT I just generated a patched plugin org.obeonetwork.m2doc starting from release tag 4.0.2.
The classes modified to solve it are:

• BookmarkManager.java
• RawCopier.java
• M2DocEvaluator.java

I also merged Bookmark services from commit 7d5884a.
It solves the issue I experienced, but of course an extensive test is recommended.
Happy to contribute

[pgesposito]

I was also able to implement issue #561.

[ylussaud]

Can you open a pull request (one per issue) so I can have a look ?

[pgesposito]

I am not enabled to push a local branch to create a pull request. Should I start from a fork?

[ylussaud]

Yes you can fork the M2Doc repository and push your branch on your fork then contribute the pull request from there.

• fork the repository
• add the forked remote to your current clone
• push your branch to the forked remote
• go to github to open the pull request from your fork

[pgesposito]

I created a pull request but I am a bit lost, because it is applicable to release tag 4.0.2 but file changes reported are referred to the master head.

[ylussaud]

At the top of your pull request you can select the branch you want to contribute to (see this documentation). By default it's master.

[pgesposito]

Selected branch 4.0.x

[ylussaud]

That's better, thank you. I'll check at least the patch for #561 quickly, hopefully next week...