Need clarification about fi_shutdown() and FI_SHUTDOWN events - Issue#10405

[syskovprdap]

(same question was asked on Slack)

Hi, I need some clarification about fi_shutdown() calls and related FI_SHUTDOWN events (my questions mainly concern on TCP and Verbs providers):

1. What is the regular flow to shutdown a connection that is establishing (FI_CONNECTED event not received yet)? Is the application expected to call fi_shutdown() or directly close the endpoint with fi_close() if the connection is establishing?
2. Is it safe to tear down the connection (fi_shutdown()) and immediately close the endpoint with fi_close() without waiting for the FI_SHUTDOWN event? Is there any risk of side effects, especially with the verbs provider (Inifniband/RoCE) where the same QP number is reused before timewait timeout expires? (@shefty, I would like to have your opinion on this point: https://general.openfabrics.narkive.com/N3q1lIZS/ofa-two-questions-about-rdma-cm-event-timewait-exit-and-the-timewait-state)
3. AFAIK, Verbs would emit a FI_SHUTDOWN (a.k.a RDMA_CM_EVENT_DISCONNECTED) event only when the connection was previously connected. Assuming that the application is required to the wait for the FI_SHUTDOWN event after fi_shutdown() (see point 2.), how would the application know the real EP state as seen by rdma_cm to determine if a FI_SHUTDOWN event will eventually be emitted? My concern is that the FI_CONNECTED event could be buffered in the EQ, causing the application and the verbs provider to have a different vision of the EP state.