SHA1/SHA224/SHA256 state layouts are not memcpy-safe

In my Perl bindings for libtomcrypt (CryptX), I support cloning digest/MAC contexts.

For a long time, I treated hash_state / hmac_state as effectively copyable by value, but with the current SHA1 and SHA224/SHA256 state layouts this is no longer that simple.

The issue is that these states now contain an internal pointer (state) which points into aligned storage inside the same struct (state_buf). After a raw struct copy, the cloned object still points into the original object’s buffer.

One "solution" would be to remove the clone() method from my Perl bindings, but before doing that I wanted to ask whether there is any chance that these state structs will become memcpy-friendly again.

As that mechanism exists for a reason, I guess there's not much that can be done to make it memcpy friendly again. Or is there and I simply didn't think of it yet!?

We could expose a clone() function for the state, or provide an API that repairs the state after copy ...

BTW this applies to AES as well...

We could expose a clone() function for the state, or provide an API that repairs the state after copy ...

A clone() API would be useful, Dropbear memcpys hash_state too. Would be worth highlighting the API change in release notes since code will still compile but won't work properly.

Oh, that is sad, LibTomCrypt project introduced these changes because of my changes about x86 optimization of those hash functions. Having the state being memcpy-able is very valuable. Surely, we could preserve this high value for the end users, by sacrificing a little bit of convenience on the library side.

It could be done by many ways. One such idea is: Add the single line of md->sha1.state = LTC_ALIGN_BUF(md->sha1.state_buf, 16); as the first line of the sha1_process and the sha1_done functions. There should be almost no performance penalty as this line should expand into simple bitop on the CPU side.

FYI I've the clone API done, but not pushed yet.

Add the single line of md->sha1.state = LTC_ALIGN_BUF(md->sha1.state_buf, 16); as the first line of the sha1_process and the sha1_done

FMU That alone won't help, because you have the intermediate state to be copied.

The idea is good!

What could work is to check the alignment each time these two functions are called, and then recover the values from the buffer with the original alignment if it differs.

I'll try this tomorrow!

Oh, sorry. My other idea was to store an integer there, instead of the pointer. The integer would track how many bytes of state_buf need to be "sacrificed" in order to obtain aligned buffer. Before each *_process and each *_done there would be an additional memmove.

Other idea is to let the state to be non-aligned. Then modify the implementation, so that it loads the state from un-aligned buffer, does the computation, and finally stores the state into un-aligned buffer. This change would "just" replace any calls to _mm_load_si128 by calls to _mm_loadu_si128 instead. Similar with _mm_store_si128.

Make SHA1 state `memcpy()`'able again.