UCP/PROTO: Consider RNDV_PERF_DIFF

[ivankochin]

What?

Applies UCX_RNDV_PERF_DIFF setting effect for protov2.

Why?

This control effect was removed during introducing perf-factors logic. This PR brings it back.

That how it looks like with that patch:

[ivankochin]

@yosefe @brminich currently patch improves all RNDV protocols, but maybe it worth to think about applying this change to RNDV-get/put only since there can be cases when RMA is unsupported for some reason and on big sizes there would be comparison between Eager and RNDV through AM and BW of latter would be slightly higher due to RNDV_PERF_DIFF. WDYT?

[brminich]

@yosefe @brminich currently patch improves all RNDV protocols, but maybe it worth to think about applying this change to RNDV-get/put only since there can be cases when RMA is unsupported for some reason and on big sizes there would be comparison between Eager and RNDV through AM and BW of latter would be slightly higher due to RNDV_PERF_DIFF. WDYT?

I’d leave it for all protocols for simplicity. Additionally, we have PPLN protocols, and even AM-based RNDV may be preferable to eager due to its lower memory consumption.

[ivankochin]

@yosefe @brminich currently patch improves all RNDV protocols, but maybe it worth to think about applying this change to RNDV-get/put only since there can be cases when RMA is unsupported for some reason and on big sizes there would be comparison between Eager and RNDV through AM and BW of latter would be slightly higher due to RNDV_PERF_DIFF. WDYT?

I’d leave it for all protocols for simplicity. Additionally, we have PPLN protocols, and even AM-based RNDV may be preferable to eager due to its lower memory consumption.

I thought AM-based RNDV and PPLN protocols consume same amount of memory as eager, aren't they? Do AM-based protocols send message directly to user-provided buffer?

[brminich]

@yosefe @brminich currently patch improves all RNDV protocols, but maybe it worth to think about applying this change to RNDV-get/put only since there can be cases when RMA is unsupported for some reason and on big sizes there would be comparison between Eager and RNDV through AM and BW of latter would be slightly higher due to RNDV_PERF_DIFF. WDYT?

I’d leave it for all protocols for simplicity. Additionally, we have PPLN protocols, and even AM-based RNDV may be preferable to eager due to its lower memory consumption.

I thought AM-based RNDV and PPLN protocols consume same amount of memory as eager, aren't they? Do AM-based protocols send message directly to user-provided buffer?

In case of tag matching API eager messages are queued on the receiver until receive operation is invoked. Such eager fragments may consume significant amount of memory on the receiver. With RNDV data transfer is started only when receiver is ready