You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
This issue hdmf-dev/hdmf-zarr#141 discusses differences in performance when reading from an indexed column in a DynamicTable, i.e., when we have a ragged array defined via a VectorData and VectorIndex
spike_times = nwbfile_read.units.spike_times[:] which reads reads all values from the VectorData but does not create a ragged array
spike_times = nwbfile_read.units['spike_times'][:] which reads data using VectorIndex (which in turn reads from VectorData to create a ragged array
It would be useful to benchmark the difference in performance and to see if the read of ragged arrays could be optimized. This difference in performance is likely exasperated when doing remote read, because VectorIndex likely makes many read requests to VectorData to read the different segments, rather than loading the data first into memory and then segmenting the array.
The text was updated successfully, but these errors were encountered:
This issue hdmf-dev/hdmf-zarr#141 discusses differences in performance when reading from an indexed column in a
DynamicTable
, i.e., when we have a ragged array defined via aVectorData
andVectorIndex
spike_times = nwbfile_read.units.spike_times[:]
which reads reads all values from theVectorData
but does not create a ragged arrayspike_times = nwbfile_read.units['spike_times'][:]
which reads data usingVectorIndex
(which in turn reads fromVectorData
to create a ragged arrayIt would be useful to benchmark the difference in performance and to see if the read of ragged arrays could be optimized. This difference in performance is likely exasperated when doing remote read, because
VectorIndex
likely makes many read requests toVectorData
to read the different segments, rather than loading the data first into memory and then segmenting the array.The text was updated successfully, but these errors were encountered: