Inquiry on CEBRA's Decoding Performance with Reordered Data

[Silver-Hart]

Hi, thank you for developing this remarkable tool!

Recently, I have been utilizing CEBRA to analyze neuronal population representations to investigate whether neurons can effectively encode object locations. I have been using the distance of the animal to the object as the behavioral label in my analysis.

During my experiments, I observed some intriguing results when decoding the object distance through dimensionality reduction of the manifold:

1. I trained the model using natural neuronal firing patterns and corresponding labels. The test set using natural patterns showed a decoding ability with an R² of 0.6
2. I trained the model with neuronal firing patterns reordered by label magnitude. The test set using natural patterns showed a slight decrease in decoding ability, with an R² of 0.5
3. Embedding reordered data into the model trained with reordered data significantly improved decoding performance, with an R² of 0.8
4. Embedding reordered data into the model trained with natural temporal patterns also enhanced decoding performance, with an R² of 0.7

Additionally, I noted the following:

a. Under normal conditions (result 1), if the model has no decoding ability for the label, reordering still does not confer decoding ability.

b. A model trained on firing data reordered based on shuffled labels also lacks decoding ability, even though a model trained on unshuffled, natural data can decode.

These observations suggest that reordering the neuronal population firing patterns based on the labels, despite disrupting the temporal firing relationships within the manifold, significantly enhances the decoding ability of the manifold. I am curious about the underlying reason for this. Could it be due to data leakage caused by the reordering of the test set during the manifold embedding stage, or is there another explanation?

The above findings were consistently observed across the official rat datasets 'achilles', 'buddy', 'cicero', and 'gatsby'. When decoding the animal's position on a linear track, the results from the reordered data consistently outperformed those from the natural data.

All tests were conducted using the time_series_rolling_window method with five-fold cross-validation.

The model parameters are as follows:

model_architecture: 'offset10-model'
batch_size: 64
learning_rate: 0.0003
temperature: 1.0
output_dimension: 3
max_iterations: 5000
distance: 'cosine'
conditional: 'delta'
device: 'cuda_if_available'
verbose: True
delta: 0.2

Attached is my test code.
Test.zip

I would greatly appreciate any insights or explanations you could provide regarding these results.

[BrainofZedZed]

Not a real answer, but I'll chime in that I observed this with my own data. Decoding using embeddings trained with shuffled labels performed very well, or even better than decoders using neural signals. My hand-waving explanation to myself was that the even with the shuffled labels, the components pulled out by the ICA can still discriminate the labels. Perhaps because the labels are shuffled, the contrastive learning fails to find common attributes in the data and therefore doesn't alter the ICA too much?

[stes]

Hi @Silver-Hart , I would be happy to look into this more over the next days. Would it be possible to give an update from your end, any new insights I should be aware of? Will have a look at the code you shared otherwise.