Does Khepri support limiting the number of returned entries?

[gomoripeti]

I just had a quick glance at the khepri:get_many family of queries and I wonder if it is possible to limit the number of returned entries, and use continuation/paging similar to mnesia:select/4 and mnesia:select/1?

[dumbbell]

Currently, there is no such mechanism.

A not-that-close alternative would be to work with khepri:fold() or khepri:foreach(). I guess I would build such a feature on top of them.

With concurrent updates, would you expect a paging mechanism to work on a snapshot and thus be unaffected by concurrent changes?

[gomoripeti]

Thanks for your response. No, In my case I wouldn't expect the paging to work on a consistent snapshot, similarly to ets table traversals. khepri:fold() is a good suggestion.
My use case would be just to limit the amount of memory that is used at once by a client process independent of khepri table size. If for example I would like to write the table content to files of 1000 element batches, I could accumulate a counter and max 1000 elements with khepri:fold() and write to file periodically (if I understand correctly) Is the callback function executed within the khepri process? So in the file writing example (which is just side effects) no terms would be copied to the calling process?

[dumbbell]

If for example I would like to write the table content to files of 1000 element batches, I could accumulate a counter and max 1000 elements with khepri:fold() and write to file periodically (if I understand correctly)

Yes

Is the callback function executed within the khepri process? So in the file writing example (which is just side effects) no terms would be copied to the calling process?

Yes, the callback is executed by the Ra server running the Khepri state machine code to be exact. This is an issue because it may block other operations for an extended period of time. I filed #171 a few months ago to address this.