[kaleb-himes]

Description

sha512: SHA-512/224 FinalRaw and Transform no longer modify state or input

Sha512FinalRaw: for SHA-512/224 (28-byte digest, not a multiple of 8) the little-endian path byte-reversed sha512->digest in place before copying it out and never reversed it back, so wc_Sha512_224FinalRaw() corrupted the running hash. Any Update after FinalRaw then produced the wrong digest. It now reverses into a local copy, copies digestSz bytes out and zeroes the copy, so sha512->digest is never modified. 5172ff7 (v5.8.0) removed an earlier local copy, citing a gcc SEGV on AVX-512 -march=native builds. That SEGV occurs only under AddressSanitizer with detect_stack_use_after_return=1 (a gcc code-generation issue, GCC Bugzilla 110027), never in a non-sanitizer build. ee2e634 later fixed the multiple-of-8 digest sizes but left SHA-512/224. The v5.2.1 and v6.0.0 FIPS modules predate it and are not affected.

wc_Sha512Transform: the input block is const, but on little-endian builds
without the Intel AVX path the function byte-reversed it in place. Each path
now leaves the caller's block and the hash state alone:

• generic C: copies into sha->buffer and reverses there, as Sha512Update
  does, then restores sha->buffer.
• ARM, RISC-V and PPC asm: pass the block unchanged on both endiannesses,
  as Sha512Update's bulk path does. On ppc64le this also fixes a wrong
  digest: the block was reversed before PPC64 asm that already loads it
  with ldbrx, so it was reversed twice. On big-endian the old code reversed
  the block once, so Transform disagreed with Update.
• ARMv5 (WOLFSSL_ARM_ARCH < 6) and RISC-V vector crypto
  (WOLFSSL_RISCV_VECTOR_CRYPTO_ASM): copy into the local aligned buffer
  first, because the ARMv5 base asm loads words with LDR and the RISC-V
  vector asm loads with vl8re64.v, which RVV 1.0 section 7.9 allows to trap
  on a misaligned address.
• AArch64: saves and restores sha->buffer around the transform, because the
  C CPU step (Transform_Sha512_C) writes it. Before, Transform overwrote
  input pending from an earlier Update.
  No path reads or writes the caller's block through a word64 pointer.

Testing

• test_wc_Sha512Transform: checks the caller's block is unchanged after
  Transform. Skipped for FIPS modules before v7.0.0: the frozen v5 and v6
  sha512.c still reverse the caller's block when the Intel AVX path is not
  used (seen on ARMv8 with --enable-fips=v5).
• test_wc_Sha512_224FinalRaw: hashes "ab", reads FinalRaw, hashes "c", and
  checks the result is SHA-512/224("abc") from the FIPS 180-4 examples.

Verified (--enable-opensslextra, master sha512.c as the control):

• x86_64 non-FIPS: master fails test_sha512.c:279 and :360; this passes;
  make check passes.
• aarch64 macOS (Apple M1 Max, --enable-armasm): master fails :279 and :360;
  this passes.
• aarch64 macOS, --enable-armasm with WOLFSSL_ARMASM_NO_NEON (C CPU step):
  Update("ab"), Transform, Final now matches the x86 and NEON builds;
  master gives a different digest.
• aarch64 big-endian (qemu, Arm GNU Toolchain 14.2 aarch64_be): Transform
  and Update of the same block now give the same state with both the NEON
  asm and the C CPU step; before they differed.
• riscv64 (BeagleV, --enable-riscv-asm): master fails :279 and :360; the
  Transform changes pass.
• riscv64 vector crypto (qemu -cpu max, --enable-riscv-asm=zv,zvkb,zvkned)
  and base (--enable-riscv-asm): SHA-512 tests pass, and an aligned and a
  misaligned Transform of the same block give the same digest.
• ppc64le (qemu, --enable-ppc64-asm and =inline): before, the SHA-512,
  SHA-512/224 and SHA-512/256 Transform tests got the wrong digest; this
  passes.
• ppc64 big-endian (qemu, --enable-ppc64-asm): passes, unchanged.
• ARMv5TE (qemu, --enable-armasm, -march=armv5te): passes.
• --enable-fips=v5 (v5.2.1 module): Transform test passes with the version
  gate and fails at test_sha512.c:278 without it.
• test_wc_Sha512Transform also compares Update("ab"), Transform,
  Update("c"), Final against Transform, Update("abc"), Final, so Transform
  must keep input pending from an earlier Update.
  • AVX-512 (AWS Xeon Platinum 8488C and EPYC 9R14): gcc 11.5, 12.4, 13.3 and 14.2 and clang 15, at -march=native -O2, -O3 and -O3 -mprefer-vector-width=512. Every SHA-512 family FinalRaw at output offsets 0 to 63 and context offsets 0 to 56 shows no fault and no state change, with each SHA-512 lane (AVX2+RORX, AVX2, AVX1+RORX, AVX1, C) forced and confirmed. The same holds in the Linux 6.12 kernel module self-test (C and AVX2). Under the original ASan sanitizer recipe the new FinalRaw also shows no fault on gcc 13.3, 14.2 and 15.2. Controls: an aligned zmm store to a misaligned address faults on that hardware, and master fails the SHA-512/224 state check.

Fixes Fenrir:

• F-17180: wc_Sha512Transform byte-swaps the caller's const input block in
  place on little-endian builds.
• F-3343: wc_Sha512Transform performs unaligned word64 byte-reversal on the
  caller's data pointer.
• F-17178: ppc64le wc_Sha512Transform reverses the block twice.

NOTE: Not changed here:

• --enable-ppc64-asm=crypto on ppc64le loads the message with lxvd2x and no
  little-endian byte swap, so SHA-256 and SHA-512 KATs fail. The asm is
  generated from wolfSSL/scripts, so the fix is in a separate pair: branch
  PQ-FS-2026-Part4-LabPrep-ppc64-sha2-crypto-le (scripts a879f1dd, wolfssl
  fb82943).
• Big-endian aarch64 --enable-armasm fails the SHA-256 and SHA-512 KATs on
  master; that is not addressed here.

Related PR's:
#11999
https://github.com/wolfSSL/scripts/pull/706

Checklist

• added tests
• updated/added doxygen
• updated appropriate READMEs
• Updated manual and documentation

[github-actions]

MemBrowse Memory Report · Explore what changed →

gcc-arm-cortex-m4-pkcs7

• FLASH: .text +64 B (+0.0%, 222,804 B / 262,144 B, total: 85% used)

zephyr-frdm_rw612

• FLASH: log_const_area -8 B, rodata -8 B, text +1,592 B (+0.6%, 284,040 B / 67,108,864 B, total: 0% used)

No memory changes detected for:

• gcc-arm-cortex-m0plus
• gcc-arm-cortex-m3
• gcc-arm-cortex-m4
• gcc-arm-cortex-m4-baremetal
• gcc-arm-cortex-m4-crypto-only
• gcc-arm-cortex-m4-dtls13
• gcc-arm-cortex-m4-min-ecc
• gcc-arm-cortex-m4-openssl-compat
• gcc-arm-cortex-m4-pq
• gcc-arm-cortex-m4-rsa-only
• gcc-arm-cortex-m4-sp-math
• gcc-arm-cortex-m4-tls12
• gcc-arm-cortex-m4-tls13
• gcc-arm-cortex-m7
• gcc-arm-cortex-m7-pq
• gcc-arm-cortex-m7-tls13
• stm32-sim-stm32h753

---

_{Catch memory regressions in CI with MemBrowse. Free for open source.}

[kaleb-himes]

@claude review please

[kaleb-himes]

@claude review again please.

[kaleb-himes]

Rebased on master to resolve CI issues that had landed in master.

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11983

Scan targets checked: none
Unchanged since last review (not re-run): wolfcrypt-src, wolfcrypt-bugs, wolfssl-bugs

Fenrir result: Approved ✅

No new issues found in the changed files.

Advisory only — this automated result does not count as a GitHub approval.

Review tier: Lite

[Copilot]

🟢 Approval recommended

No active review issues remain for the SHA-512 state-preservation and input-handling changes.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🟢 Approval recommended

No active review issues are recorded for the SHA-512 state and input preservation changes.

No open issues 🎉

1 resolved since last review

• Avoid reintroducing the unaligned scratch-array workaround

_{Review effort: Balanced}

[wolfSSL-Fenrir-bot]

Fenrir Automated Review — PR #11983

Scan targets checked: wolfcrypt-src, wolfcrypt-bugs, wolfssl-bugs
Coverage: 2 of 2 in-scope changed file(s) opened by the reviewer

Fenrir result: Approved ✅

No new issues found in the changed files.

Advisory only — this automated result does not count as a GitHub approval.

Review tier: Max