What are the validity requirements of wide pointers/references with dyn Trait tail?

[RalfJung]

This is the last remaining part of #166, now that #510 has been resolved.
Note that the safety invariant for both of these is pretty much set already since casts from *const dyn Trait to *const dyn Supertrait are safe -- that means the vtable pointer must be valid for the given trait, even on raw pointers.

For the validity of the metadata in references, there seems to be general consensus that it is the same as the safety invariant: the vtable must be a vtable for the right trait (and an arbitrary type).

So the open question is what to do with the validity of metadata in raw pointers. The two contradicting goals pursued by different people here are:

• avoid the confusion that can arise when validity and safety invariant diverge
• avoid the pitfall of a NULL vtable pointer (or some other sentinel value) being insta-UB

This was discussed in Zulip somewhat recently. My proposal was to take a maximally liberal validity invariant, and allow any vtable pointer, but several people had concerns that it would be confusing to allow allow a null (or general, unsafe) vtable to exist but then make it UB to cast that pointer or use it as a dyn receiver, even though that operation is safe. One possible compromise was to explicitly allow only null and then make that actually safe by adding checks on casts and dyn calls -- but that would go against the idea of using raw pointers in cases where performance matters and safe code has too many checks.

[RalfJung]

One interesting sub-question to consider here is the exact definition of "the right trait". Currently in Miri, "the right trait" is defined as "has the same principal trait", where the "principal" trait is the trait that is left after removing all auto traits.

So for instance, transmutes between dyn Debug and dyn Debug+Send are allowed, but transmutes between dyn Debug and dyn Display are not. dyn Debug + Send vs dyn Send is also not allowed, because the former has Debug as the principal trait while the latter does not have a principal trait.

It seems to me that the most consistent and conservative option right now would be to require full equality of the trait list, i.e. to also rule out transmutes between dyn Debug and dyn Debug+Send. Though that does raise the question whether order should matter... arguably, it should not.