Working Workflow Example

[rosecers]

Some notes:

• because this is living in the kernel-tutorials repo, I expanded the sections on methods covered in kernel tutorials, but left the methods not covered in kernel-tutorials somewhat cursory. In other words, cells for librascal and chemiscope are somewhat succinct, although this can be brought up for debate.
• I've left the notebook unstripped so that you don't have to run it to see the output. We can strip upon merge.
• There is the resulting chemiscope file for viewing under notebooks/kpcovr-chemiscope.json

[Luthaf]

I have mostly small comments, as well as a higher level one.

This notebook is very good as a "you have done all of kernel-tutorials, now here is a end-to-end example of how you could use it" notebook, which is a different use case from the "here is how you get from a structure to a chemiscope input" which I had in mind for lab-cosmo/chemiscope#114.

In particular, the use of (PCov-)FPS, (PCov-)CUR, and sparse methods is adding a lot of complexity for people that might just want to try out chemiscope quickly. The notebook also assumes knowledge of SOAP and descriptors, and do not introduce the reason behind dimensionality reduction.

I think the question here is how much do we want to couple this to the rest of kernels-tutorials/skcosmo. Should someone be able to read/use this notebook without reading the rest of kernel-tutorials?

[Luthaf]

This is somehow self-explanatory, but a short comment/docstring for parameters would be nice

[rosecers]

Addressed in most recent commit!

[Luthaf]

Suggested change

	"**Because we are demonstrating with an environment-centred property, we pull the properties from the frame.arrays object.**"
	"**Because we are using environment-centred properties in this demonstration, we pull the properties from the frame.arrays object.**"

[rosecers]

Addressed in most recent commit!

[Luthaf]

This comment do not match the code

[rosecers]

Addressed in most recent commit!

[Luthaf]

Maybe add a link to said tutorial?

[rosecers]

Addressed in most recent commit!

[Luthaf]

Suggested change

	"# # Import SOAP vectors from numpy file\n",
	"# # read/load SOAP vectors from numpy file\n",

import has a separate meaning in Python, so I think something like read or load would remove a possible source of confusion.

[rosecers]

Addressed in most recent commit!

[Luthaf]

This part of the code is not very clear. Amongst other things, Phi is not defined, the 1e-12 in S = np.maximum(S, 1e-12) is a bit arbitrary, and the use of SVD to build a normalization matrix is not explained. Is this talked about in the previous notebooks?

[Luthaf]

not sure what the comment means. How can I know/make sure Y has been estimated, and what does "estimate Y" means in this context?

If this is all material covered in previous notebooks, then my question becomes how much do we want to couple this notebook to previous ones?

[rosecers]

Addressed in most recent commit!

[Luthaf]

Suggested change

	"# Exporting to a Chemiscope"
	"# Exporting to chemiscope"

[rosecers]

Addressed in most recent commit!

[Luthaf]

Suggested change

	"We'll write one output combining the training set and the testing set."
	"Chemiscope is an online interactive visualizer that allows you to look at the projection in relation with the structures. Here, we'll create a chemiscope input file combining the training set and the testing set."

[rosecers]

Addressed in most recent commit!

[Luthaf]

this is done automatically in chemiscope.write_input if you pass

chemiscope_properties = {
    "kpcovr": {
        "target": "atom", 
        "values": T[:, :kpcovr.n_components]
    },
    "cur_feature": {
        "target": "atom", 
        "values": cur_features[:, :n_cur]
   },
    **{
        prop_name: {"target": target, "values": Y[:, i]}
        for i, prop_name in enumerate(properties)
    },
}

[rosecers]

Addressed in most recent commit!

[rosecers]

@Luthaf As for your larger comment, I think that's the conversation you, me, @max-veit, and @ceriottm need to have. If we want a front-to-back ase-to-chemiscope pipeline, where should it live? Is it meant to be internal to the group or publicly available?
As I said, I went heavy on the scikit-cosmo-like functionalities because we're in kernel-tutorials. A possible solution is to have three notebooks all doing the same thing, but with different emphases (librascal, scikit-cosmo/KT, chemiscope), living in the three packages. This is a bit dangerous though, due to the constantly updating nature of the packages.

I made this notebook 1) to start this conversation, 2) because I think regardless of the answer, it's a nice "bow" to K-T, and 3) because until we can figure out a more centralized example, it can serve as a "until-then" solution for group members wanting a front-to-back solution. What do you think?

[Luthaf]

Fully agree on all points, and we can move forward with this notebook as a "bow" for kernels-tutorials regardless of the outcome of the other discussion!

Personally, I think having a separate "this is the simplest way to create a chemiscope input" example would be very valuable; and could point to the notebook introduced in this PR as a good point of "here is how you can improve the projection a lot".