Taking some notes from investigation:
We already have logic to clear unmanaged hoisted locals when we exit scopes. But that logic has a number of limitations:

1. we don't clear top-level locals on normal exit
2. we don't clear any locals on yield break
3. we don't clear any locals on early exit (from yield return, ie. foreach (var i in coll) { break; })
4. we don't clear any locals on throw

So the current design only works for nested locals on normal exit from the scope...

static IEnumerable<int> Produce()
{
    {
        string hoistedNestedLocal = "";
        // existing clearing logic doesn't work for:
        // `yield break;`
        // `yield return;` and the caller breaks `foreach (var i in coll) { break; }`
        // `throw`
    }

    string hoistedTopLevelLocal = "";
    // existing clearing local doesn't work for:
    // normal exit
    // `yield break;`
    // `yield return;` and the caller breaks `foreach (var i in coll) { break; }`
    // `throw`
}

From discussion with Jared, letting go of references is best effort. It doesn't need to be complete/perfect. In particular, we don't want to pay the cost of an extra try/finally to achieve it. So I'm not proposing to address scenario 4 (thrown exception) perfectly. If an exception is thrown during disposal, the hoisted locals won't be cleared.

We discussed two options for iterators with Aleksey, which would address all non-exceptional scenarios, for both top-level and nested locals:

1. clear hoisted all locals as last thing in Dispose (could be made cheap by holding all enumerator state with a nested struct)
2. replicate block structure in Dispose, clear locals in that scope at end of block

For async-iterators, there are similar alternatives which could work.