Changes to systemd journal parser to handle corrupt and binary data

[kev365]

Description

Makes the systemd journal parser resilient to the damaged, rotated and unclean journals that matter most in forensics. Three related robustness fixes:

• Continue past a corrupt entry instead of returning. The entry loop in ParseFileObject did return on the first ParseError, abandoning every entry after the first unparsable one. It now does continue, matching systemd's reader guidance to skip corruption and recover as much surrounding data as possible.
• Decode field data at the byte level. _ParseJournalEntry did event_data.decode("utf-8") with no error handling; a non-UTF-8 field value (the on-disk format permits binary DATA payloads) raised an uncaught UnicodeDecodeError — a ValueError, so the surrounding except errors.ParseError never caught it. The split-and-decode is extracted into _SplitFieldData, which splits on the first = byte and decodes with errors="replace" (event-data values must be str).
• Wrap XZ and LZ4 decompression in try/except → ParseError, as ZSTD already was, so corrupt compressed data raises a ParseError instead of an uncaught library exception.

Related issue: fixes #5109

Testing

• testParseDirty updated: the parser now reports all 46 corrupt entries in the unclean tail (1 corrupt entry pointer + 45 unused objects) instead of only the first; the event count (2211) and the first-warning message are unchanged. The 46 = 1 + 45 breakdown is asserted so the count is documented rather than magic.
• Added testSplitFieldData covering a text value and a non-UTF-8 value (→ U+FFFD replacement).
• Validated end-to-end against journalctl: a journal written by systemd-journal-remote with a MESSAGE containing non-UTF-8 bytes parses to a body with U+FFFD, matching journalctl --output=json's byte-array rendering.
• Existing clean-journal tests (testParse, testParseLZ4, testParseCompactZSTD) unchanged and passing; Black + pylint clean.

Open question for the reviewer

On the dirty-journal sample the extraction-warning count rises 1 → 46 because every corrupt tail entry is now surfaced rather than only the first. If that's too noisy, I'm happy to collapse a run of trailing Unsupported object type: 0 entries into a single "reached corrupt tail" warning — let me know your preference.

Checklist

• No new dependencies are required or l2tdevtools has been updated.
• Test data has a Plaso compatible license. (No new test data added.)
• Reviewer assigned.
• Automated checks (GitHub Actions, AppVeyor) pass.

[codecov]

Codecov Report

❌ Patch coverage is 88.46154% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 85.09%. Comparing base (680976f) to head (4257669).
⚠️ Report is 3 commits behind head on main.

Files with missing lines	Patch %	Lines
plaso/parsers/systemd_journal.py	88.46%	3 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #5113   +/-   ##
=======================================
  Coverage   85.09%   85.09%           
=======================================
  Files         455      455           
  Lines       40393    40411   +18     
=======================================
+ Hits        34371    34387   +16     
- Misses       6022     6024    +2     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[joachimmetz]

This will silently replace "invalid" characters let's make sure to set an extraction warning if this situation occurs

[kev365]

Thanks Joachim, here is a proposed adjustment for the extraction warning:

_SplitFieldData now decodes each field via a new _DecodeFieldData, which produces an extraction warning ("Unable to decode journal field data as UTF-8, replaced invalid characters") when invalid UTF-8 is replaced, rather than replacing silently (dc23560). The parser mediator is threaded down (_ParseJournalEntry → _SplitFieldData → _DecodeFieldData). testSplitFieldData asserts the warning fires on a non-UTF-8 value and not on a valid one.

[joachimmetz]

Unfortunately decode("utf-8", errors="replace") uses the Unicode placeholder character which makes it very hard to determine what the original data was from the extracted data.

It might be better to omit the log line than present an altered one, or better to "escape" the unsupported byte values

[joachimmetz]

errors="backslashreplace" might be more appropriate

[joachimmetz]

LGTM