DDA identification - ?

[mlocardpaulet]

Since we decided to give up the entrapment strategy, we discussed a different type of DDA identification module:
It would be nice to plot overlap of identification from pairs of searches. It is too complicated to have it as a main figure: we would need to select pairs. So we think of a main figure to select from

• Number of ID vs rank, we could have a slider that select the peptides matched to proteins identified with 1, 2, etc.. peptides; or 2 plots (proteins with >1 proteotypic peptide or all)
• Number of ID vs AUC (based on number of ID per FDR). Apparently we only need the q-value per psm in the output.
  What do we count? Ions? Peptidoforms? PSMs would be easier, but what do we do if we have multiple PSMs per spectrum: We keep the best one.
• Number of peptidoform sequence vs number or validated spectra
  What file to use? Human sample?

[julianu]

Dataset for testing:
https://ftp.pride.ebi.ac.uk/pride/data/archive/2022/02/PXD028735/LFQ_Orbitrap_DDA_Human_01.raw

[julianu]

We should record the way of post-processing

• "simple" target decoy, percolator, other re-scoring

[veitveit]

Shouldn't be the level (PSM, peptidoform, protein) of the validation be on what was uploaded?

For example, if you want to check for protein IDs, then the user should upload a table with protein accession names + their scores.

Otherwise, ProteoBench carries out part of the data analysis, and thus would influence the outcome by whatever choices have been made.

[julianu]

I would not really care about proteins for this module.
We definitely need the user to upload PSMs, I would suggest already with calculated q-values. Alternatively, we could also simply accept the PSMs of targets, decoys and the search engine score and calculate the scores ourself.

The benefit with already calculated q-values would be, we can also accept re-scored or percolated data.
But having a little "sanity check" would be good. Otherwise, one could just upload a totally synthetic, overtuned dataset.

[julianu]

A first impression of possible metrics (Dataset not run with the discussed FASTA, but using cRAP for contaminants)

[julianu]

For now, the upset plot uses all datapoint. But these should probably be selected and allow only up to 5 datasets or something. Otherwise, this will get extremely messy.

[mlocardpaulet]

I would be more interested in the overlaps between all and all but one workflow. I personally would focus on these (if we have to choose).

[julianu]

Updates of the plots:

Plots with AUC from more "real" ROC (normalized by number of IDs per searchengine)

And a plot of the number of IDs against q-value (not normalized)

... and normalized