Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chunked kernels #19

Open
keewis opened this issue Jun 14, 2024 · 0 comments
Open

chunked kernels #19

keewis opened this issue Jun 14, 2024 · 0 comments

Comments

@keewis
Copy link
Member

keewis commented Jun 14, 2024

While building the kernel, we're currently a bit wasteful, and for bigger kernels the intermediate arrays may quickly become too huge to fit into memory.

We work around this by cleverly chunking the cell ids / neighbours / angular distances, but at the moment we still load all the coordinates / data into memory when creating the sparse matrix.

However, it should be possible to also produce a chunked sparse matrix. This will require some though, as right now we use the cell ids as indices into the sparse array (i.e. the kernel producing output cell id 201 is at row 201 in the kernel matrix). Thus, we need to figure out a way to translate the cell ids to indices into the chunk.

As an added bonus, this may make the regional kernels easier to construct (see #18)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant