Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add FDATransform #1726

Open
3 tasks
Tracked by #1646
guarin opened this issue Nov 7, 2024 · 6 comments · May be fixed by #1734
Open
3 tasks
Tracked by #1646

Add FDATransform #1726

guarin opened this issue Nov 7, 2024 · 6 comments · May be fixed by #1734

Comments

@guarin
Copy link
Contributor

guarin commented Nov 7, 2024

Add FDATransform based on Disentangling the Effects of Data Augmentation and Format Transform in Self-Supervised Learning of Image Representations.

The different subtransforms have already been added in #1646
The goal now is to combine them all into a single FDATransform. Note that the image in the paper shows that the amplitude rescale and phase shift transforms are applied in parallel but we can implement them sequentially as the subtransforms handle the split into amplitude/phase components internally.

Tasks

Follow the BYOLTransform code on how to implement the different views and the full FDATransform: https://github.com/lightly-ai/lightly/blob/master/lightly/transforms/byol_transform.py

Each view transform must implement the following steps:
Screenshot 2024-11-07 at 10 02 15

Parameters for view 1 and 2:
Screenshot 2024-11-07 at 10 03 00

@vectorvp
Copy link
Contributor

vectorvp commented Nov 7, 2024

Hello @guarin, please assign this issue to me.

@guarin
Copy link
Contributor Author

guarin commented Nov 7, 2024

Thanks for looking into it!

@vectorvp
Copy link
Contributor

@guarin, while implementing this I see there is no probability parameters in AmplitudeRescaleTransform and PhaseShiftTransform. Should we add it or what is the workaround?

@guarin
Copy link
Contributor Author

guarin commented Nov 12, 2024

Hi, you can use the RandomApply transform from torchvision

@vectorvp
Copy link
Contributor

@guarin, please review this small PR. After I will make a PR for the current issue.

@vectorvp vectorvp linked a pull request Nov 14, 2024 that will close this issue
@vectorvp
Copy link
Contributor

@guarin, hello, please review the PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants