In the paper of Swin-UNETR and self-supervise pre-train, the relative position bias is missing

[ZefanW]

And it's vital in Swin-like models. If I have read the correct version of swinUNETR code, it actually has relative position bias.

[saidharb]

I was asking myself the same thing. In the paper in formula (2) there is no relative positional bias, even though the Swin Transformer which is the backbone uses it.