Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add NumPy 2 support #605

Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

Add NumPy 2 support #605

wants to merge 7 commits into from

Conversation

rhugonnet
Copy link
Contributor

@rhugonnet rhugonnet commented Sep 5, 2024

This PR updates GeoUtils to support NumPy 2.0.
I am not going to merge it now, as some issues arising seem to come from potentially undesirable NumPy masked-array behaviour.

Summary

The issue is the new casting rules for scalars with arrays, especially for integers, see here: https://numpy.org/devdocs/numpy_2_0_migration_guide.html#changes-to-numpy-data-type-promotion and https://numpy.org/neps/nep-0050-scalar-promotion.html#nep50 for details.

The above rules make sense, but don't seem to behave the same for masked array. Somehow, we cannot easily cast an integer with a masked array of lower integer-type anymore:
For instance:

test = np.array([1, 2], dtype="uint8")
test += 5

runs fine.
But:

test = np.ma.masked_array([1, 2], dtype="uint8")
test += 5

yields:

  File "/home/atom/miniconda3/envs/geoutils-dev/lib/python3.11/site-packages/numpy/ma/core.py", line 4422, in __iadd__
    self._data.__iadd__(other_data)
numpy._core._exceptions._UFuncOutputCastingError: Cannot cast ufunc 'add' output from dtype('int64') to dtype('uint8') with casting rule 'same_kind'

Additionally

With the new scalar casting behaviour, we might have to relax the current behaviour of Raster.data.setter() expecting the same data type, otherwise this type of command to simply update the array:

rast.data +=1

might fail a lot.
We can dive into this in more details once the above exception is clarified.

Steps forward

I will open an issue on NumPy's GitHub after triple-checking if this is intended behaviour, but I don't think so.

@rhugonnet
Copy link
Contributor Author

Update: Opened issue numpy/numpy#27679

@rhugonnet
Copy link
Contributor Author

Confirmation this is a bug from NumPy, we'll have to wait until it is fixed there to update to 2.0. Will likely take a while!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant