Trait for safe all-zero byte initialization (v2)

[DanielEScherzer]

Proposal

Problem statement

When dealing with FFI types, especially those that will be initialized by code on the other side of the FFI boundary, creating a new instance of those types generally requires using unsafe { core::mem::zeroed() } at each site, unless some shortcut is added to the specific type.

It is good practice to document why code marked as unsafe is actually safe (e.g. as enforced by the clippy::undocumented_unsafe_blocks lint). Each place that the type is initialized is going to have similar documentation.

Motivating examples or use cases

In the git2 crate there are dozens of cases where std::mem::zeroed() is used for creation before initialization is done with FFI, e.g. (https://github.com/rust-lang/git2-rs/blob/da6c126f7733329d82748e710266af1c61591bdd/src/repo.rs#L2199-L2204)

        unsafe {
            let mut raw_opts = mem::zeroed();
            try_call!(raw::git_checkout_init_options(
                &mut raw_opts,
                raw::GIT_CHECKOUT_OPTIONS_VERSION
            ));
            // ...

Solution sketch

/// Indicates that a type is valid with all-zero bytes
///
/// # Safety
///
/// Marking a type as `ZeroInitable` requires that an all-zero byte-pattern
/// represent a valid instance of the type.
pub unsafe trait ZeroInitable: Sized {

    fn zeroed() -> Self {
        // SAFETY: Per the safety requirements of the trait, an all-zero
        // byte-pattern is valid for this type
        unsafe { core::mem::zeroed() }
    }

}

Alternatives

Not do this, and keep using core::mem::zeroed()

Links and related work

There was a previous similar proposal at #107 for a derive-able trait, but this proposal is different in a few ways, most notably that the trait is not derivable, and thus the compiler doesn't need any new logic or language handling, so the feedback on the prior proposal that "It runs up against lang-team concerns, like safe transmute and related efforts." does not apply. The new proposed trait is also explicitly unsafe - the existence of the trait does not affect any memory safety, and the implementation of the trait indicates that a developer has explicitly confirmed that an all-zero byte-pattern is valid, the same as calling core::mem::zeroed().

The Default trait is used to indicate that a particular byte representation is a good "default" representation, but for FFI types this may not be very applicable, especially when the initialization needs to be done with an FFI call, and rust just needs to create some instance of the object.

What happens now?

This issue contains an API change proposal (or ACP) and is part of the libs-api team feature lifecycle. Once this issue is filed, the libs-api team will review open proposals as capability becomes available. Current response times do not have a clear estimate, but may be up to several months.

Possible responses

The libs team may respond in various different ways. First, the team will consider the problem (this doesn't require any concrete solution or alternatives to have been proposed):

• We think this problem seems worth solving, and the standard library might be the right place to solve it.
• We think that this probably doesn't belong in the standard library.

Second, if there's a concrete solution:

• We think this specific solution looks roughly right, approved, you or someone else should implement this. (Further review will still happen on the subsequent implementation PR.)
• We're not sure this is the right solution, and the alternatives or other materials don't give us enough information to be sure about that. Here are some questions we have that aren't answered, or rough ideas about alternatives we'd want to see discussed.

[ChayimFriedman2]

Any proposal for such thing should explain how it relates to Project Safe Transmute and crates like zerocopy and bytemuck.

Also, without a derive, the usability is practically limited to std types - which makes this mostly not useful.

[DanielEScherzer]

Any proposal for such thing should explain how it relates to Project Safe Transmute and crates like zerocopy and bytemuck.

• Project Safe Transmute looks like it covers a whole bunch more, and the conversion from all zeros would be either with a safe_transmute() call (where you need to allocate the byte array before a transmute call, meaning it would less ergonomic/more complicated than just using core::mem::zeroed()), or an unsafe_transmute() call (which has the same issues, plus the original problem of needing to repeat a justification for why the conversion is actually safe)

• ZeroCopy: the FromZeros trait is documented as

Do not implement this trait yourself! Instead, use #[derive(FromZeros)]; e.g.:

but the derive documentation makes it clear that not everything is covered (e.g. fields that are pointers), where an all-zero byte-pattern corresponds to a null pointer, which is valid.

Even if it matches the ZeroCopy functionality, by incorporating this into the core library it makes it easier for others to use by removing the need to install an external crate

• Bytemuck: this seems to match exactly the https://docs.rs/bytemuck/latest/bytemuck/trait.Zeroable.html trait (which I didn't know about until now) but again by including this in the language it removes a barrier to use (plenty of precedent for this, looks like the most recent is Bring hashbrown::Equivalent into std #699 from a few months ago)

Also, without a derive, the usability is practically limited to std types - which makes this mostly not useful.

Why is usability limited? The trait is public, and I would envision using it alongside bindgen. E.g. for the checkout options example for git2-rs

#[repr(C)]
pub struct git_checkout_options {
    pub version: c_uint,
    // ... other fields
}

unsafe impl core::mem::ZeroInitable for git_checkout_options {}

[ChayimFriedman2]

but the derive documentation makes it clear that not everything is covered (e.g. fields that are pointers), where an all-zero byte-pattern corresponds to a null pointer, which is valid.

Nope, works fine.

by incorporating this into the core library it makes it easier for others to use by removing the need to install an external crate

Which is why (part of why) we have project safe transmute. Currently, you rely on the crates; once the effort will be finished, you might still but the analysis will be done by the compiler instead. And then it will also make more sense to have a trait for that in the standard library, based on safe transmute.

Why is usability limited? The trait is public, and I would envision using it alongside bindgen.

Because realistically, nobody is going to have dozens (if not hundreds) of unsafe impls and risk UB when they can use a crate and be 100% safe.

[DanielEScherzer]

Nope, works fine.

I guess the documentation is outdated, good to know

Which is why (part of why) we have project safe transmute. Currently, you rely on the crates; once the effort will be finished, you might still but the analysis will be done by the compiler instead. And then it will also make more sense to have a trait for that in the standard library, based on safe transmute.

Currently there are at least 2 crates that provide this functionality, so the functionality is clearly useful. Is there a timeline for the safe transmute effort to be finished?

Because realistically, nobody is going to have dozens (if not hundreds) of unsafe impls and risk UB when they can use a crate and be 100% safe.

It doesn't need to be dozens or hundreds, just the types that are needed

And the point is to take the core::mem::zeroed() calls which already risk UB, and centralize them in a single place with a common safety justification instead of needing to repeat it in each place

[tgross35]

I do wonder if any individual traits from project-safe-transmute might be possible to stabilize even if the rest still needs work. Think this may be working toward an RFC, though.

@jswrenn