Manually sync the metric in DDP training?

[Wuziyi616]

Hi! Thank you so much for the great work! I'm currently using your implemented MeanMetric in my DDP training code. The document says that it can automatically perform DDP sync. However, when I look into the code here, I don't see any code performing sync (e.g. all_gather). So I wonder should I manually call the metric.sync() function before calling metric.compute()? (I assume not) Then where are you performing the sync? Thanks!

[Wuziyi616]

BTW, if I don't want to compute every_step metric (i.e. on every batch), is calling forward the same as calling update?

[Wuziyi616]

Seems that the answer to my questions above are all True. Somehow the code can automatically handle DDP sync, amazing!