[svsm-devel] Fallible allocations and smart pointers

[Carlos López]

Hi everyone,

I'm writing this email to get some comments on the topic of fallible
allocations and smart pointers, which we briefly discussed during the
last development call.

At the moment we use smart pointer types in various places throughout
the codebase. Currently these include `Box` (regular heap
allocations), `Arc` (reference counted heap allocations) and `Vec`
(dynamically-sized arrays). In the future we might also end up using
`Rc` or some of the map and set flavors. These types are provided by
the alloc crate, which is part of the core Rust library.

Given the goals of the SVSM, we would want to be able to recover from
allocation failures, as simply panicking is not acceptable. These
smart pointer types currently do not have stable APIs with a failure
mode - e.g. the following:

     let foo = Box::new(MyType::new());

will panic if the global allocator can't reserve `size_of::<MyType>()`
bytes with the adequate alignment.

Once the `allocator_api` feature [0] is stabilized, we will be able to
do [1]:

     let foo = Box::try_new(MyType::new())?;

The feature would also let the caller choose a specific allocator if
several are available. Ideally, we would also like to enable
`no_global_oom_handling` to completely prevent all implicit panics.

In any case, `allocator_api` does not seem to be anywhere close to
being stabilized, which means we need a solution in the meantime,
ideally without going back to using the nightly toolchain.

I have looked into several options which I list below with their
respective downsides. I have not tested any of them yet.

1. Bring our own implementation from scratch: costly and error-prone.
2. fallible-alloc crate [2]: only provides `Box` and `Vec`, so we
    would need to build the remaining types (for example the upstream
    alloc crate builds `Arc` and `Rc` out of `Box`, and so on). Costly,
    but less so than a completely from-scratch implementation.
3. fallible-collections crate [3]: supports `Box`, `Vec`, `Rc` and
    `HashMap`, but not `Arc` (it is listed as deprecated [4], and I
    suspect the root issue might also affect `Rc`). Same issues as
    point #2.
4. Linux kernel model (copy a subset of the upstream alloc files):
    requires maintenance to keep up with upstream, and likely means
    only supporting a specific version of the toolchain (see [5]).

So, if anybody has any comments regarding the topic, or wants to
propose a different approach to the ones above, please do so :)

Regards,
Carlos

[0] https://github.com/rust-lang/rust/issues/32838
[1] https://doc.rust-lang.org/alloc/boxed/struct.Box.html#method.try_new
[2] https://docs.rs/fallible_alloc/0.2.0/fallible_alloc/
[3] https://docs.rs/fallible_collections/0.4.9/fallible_collections/
[4] https://github.com/vcombey/fallible_collections/issues/13
[5] 
https://github.com/torvalds/linux/commit/3ed03f4da06ede71ac53cf25b9441a372e9f2487

-- 
Carlos López
Security Engineer
SUSE Software Solutions