Fix 0.4.0dev tests #384

talonchandler · 2023-07-25T17:30:00Z

This PR against 0.4.0dev includes the test fixes that should have appeared in #381 (see #383 for CI/CD fix).

codecov · 2023-07-25T18:58:35Z

Codecov Report

Merging #384 (85b8285) into 0.4.0dev (c1ff28d) will increase coverage by 3.16%.
Report is 76 commits behind head on 0.4.0dev.
The diff coverage is 68.26%.

@@             Coverage Diff             @@
##           0.4.0dev    #384      +/-   ##
===========================================
+ Coverage      3.82%   6.98%   +3.16%     
===========================================
  Files            23      24       +1     
  Lines          4629    4450     -179     
===========================================
+ Hits            177     311     +134     
+ Misses         4452    4139     -313

Files Changed	Coverage Δ
recOrder/acq/acquisition_workers.py	`0.00% <0.00%> (ø)`
recOrder/calib/Calibration.py	`0.00% <0.00%> (ø)`
recOrder/calib/calibration_workers.py	`0.00% <0.00%> (ø)`
recOrder/io/utils.py	`0.00% <0.00%> (ø)`
recOrder/plugin/main_widget.py	`0.00% <0.00%> (ø)`
recOrder/tests/cli_tests/test_cli.py	`100.00% <ø> (ø)`
recOrder/tests/cli_tests/test_reconstruct.py	`97.14% <97.14%> (ø)`
recOrder/tests/cli_tests/test_compute_tf.py	`100.00% <100.00%> (ø)`
recOrder/tests/cli_tests/test_settings.py	`100.00% <100.00%> (ø)`
recOrder/tests/conftest.py	`100.00% <100.00%> (ø)`
... and 1 more

... and 1 file with indirect coverage changes

talonchandler · 2023-07-25T19:04:41Z

@ziw-liu I'm having trouble getting the tests to pass on windows. Any idea how we might solve this?

>           assert tf_path.exists()
  E           AssertionError: assert False
  E            +  where False = <bound method Path.exists of WindowsPath('C:/Users/runneradmin/AppData/Local/Temp/pytest-of-unknown/pytest-0/test_reconstruct0/tf.zarr')>()
  E            +    where <bound method Path.exists of WindowsPath('C:/Users/runneradmin/AppData/Local/Temp/pytest-of-unknown/pytest-0/test_reconstruct0/tf.zarr')> = WindowsPath('C:/Users/runneradmin/AppData/Local/Temp/pytest-of-unknown/pytest-0/test_reconstruct0/tf.zarr').exists

ziw-liu · 2023-07-25T19:48:33Z

I can reproduce the same error on a local Windows machine (automaton 2).

Edit: the tf.zarr is indeed missing...

ziw-liu · 2023-07-25T21:36:19Z

OK found it. Same issue as czbiohub-sf/iohub#116

ziw-liu · 2023-07-25T21:44:32Z

@talonchandler Some new warnings related to torch in addition to the overlay colormap things:

=============================== warnings summary ===============================
  recOrder/tests/cli_tests/test_compute_tf.py::test_compute_transfer
    /home/runner/work/recOrder/recOrder/.tox/py311-linux/lib/python3.11/site-packages/torch/functional.py:504: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at ../aten/src/ATen/native/TensorShape.cpp:3483.)
      return _VF.meshgrid(tensors, **kwargs)  # type: ignore[attr-defined]
  
  recOrder/tests/cli_tests/test_compute_tf.py::test_compute_transfer
  recOrder/tests/cli_tests/test_compute_tf.py::test_compute_transfer_output_file
  recOrder/tests/cli_tests/test_compute_tf.py::test_absorption_and_phase_write
  recOrder/tests/cli_tests/test_compute_tf.py::test_phase_3dim_write
  recOrder/tests/cli_tests/test_compute_tf.py::test_fluorescence_write
  recOrder/tests/cli_tests/test_reconstruct.py::test_reconstruct
    /home/runner/work/recOrder/recOrder/.tox/py311-linux/lib/python3.11/site-packages/waveorder/optics.py:321: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
      * torch.tensor(z_position_list)[:, None, None]
  
  recOrder/tests/cli_tests/test_compute_tf.py::test_compute_transfer
  recOrder/tests/cli_tests/test_compute_tf.py::test_compute_transfer_output_file
  recOrder/tests/cli_tests/test_compute_tf.py::test_absorption_and_phase_write
  recOrder/tests/cli_tests/test_reconstruct.py::test_reconstruct
    /home/runner/work/recOrder/recOrder/.tox/py311-linux/lib/python3.11/site-packages/waveorder/optics.py:370: UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
      * torch.tensor(z_position_list)[:, None, None]
  
  recOrder/tests/cli_tests/test_reconstruct.py::test_reconstruct
    /home/runner/work/recOrder/recOrder/.tox/py311-linux/lib/python3.11/site-packages/waveorder/stokes.py:2[91](https://github.com/mehta-lab/recOrder/actions/runs/5660655330/job/15336852512#step:7:92): UserWarning: To copy construct from a tensor, it is recommended to use sourceTensor.clone().detach() or sourceTensor.clone().detach().requires_grad_(True), rather than torch.tensor(sourceTensor).
      transmittance = torch.tensor(s0).clone()
  
  recOrder/tests/cli_tests/test_reconstruct.py::test_reconstruct
    /home/runner/work/recOrder/recOrder/.tox/py311-linux/lib/python3.11/site-packages/waveorder/util.py:711: RuntimeWarning: invalid value encountered in divide
      img_norm_stack[i] = img_stack[i] / uniform_filter(
  
  recOrder/tests/util_tests/test_overlays.py::test_ret_ori_overlay
  recOrder/tests/util_tests/test_overlays.py::test_ret_ori_overlay
  recOrder/tests/util_tests/test_overlays.py::test_ret_ori_overlay
  recOrder/tests/util_tests/test_overlays.py::test_ret_ori_overlay
    /home/runner/work/recOrder/recOrder/.tox/py311-linux/lib/python3.11/site-packages/recOrder/io/utils.py:271: RuntimeWarning: overflow encountered in divide
      ret_ / np.max(ret_),
  
  recOrder/tests/util_tests/test_overlays.py: 16 warnings
    /home/runner/work/recOrder/recOrder/.tox/py311-linux/lib/python3.11/site-packages/colorspacious/ciecam02.py:333: RuntimeWarning: invalid value encountered in divide
      t = (C
  
  recOrder/tests/util_tests/test_overlays.py::test_ret_ori_overlay
  recOrder/tests/util_tests/test_overlays.py::test_ret_ori_overlay
  recOrder/tests/util_tests/test_overlays.py::test_ret_ori_overlay
    /home/runner/work/recOrder/recOrder/.tox/py311-linux/lib/python3.11/site-packages/colorspacious/ciecam02.py:3[97](https://github.com/mehta-lab/recOrder/actions/runs/5660655330/job/15336852512#step:7:98): RuntimeWarning: invalid value encountered in power
      * ((27.13 * np.abs(RGBprime_a - 0.1))
  
  -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html
  
  ---------- coverage: platform linux, python 3.11.4-final-0 -----------
  Coverage XML written to file coverage.xml
  
  ======================= 29 passed, 36 warnings in 11.07s =======================

talonchandler · 2023-07-25T21:48:32Z

Thanks @ziw-liu!

Let me work through these warnings then I'll merge this.

talonchandler · 2023-07-25T22:25:37Z

I've documented the warnings into issues in the correct repositories:

waveorder

#137 - meshgrid warning with fix PR
#139 - warning about copying tensors (slightly more involved to fix)
#140 - phase normalization divide by zero

recOrder

Warnings related to colorspacious #386 - overlay warnings

None of these warnings are critical, so I vote to merge this branch into 0.4.0dev then into main.

default 4-state pol settings

61cf2dd

talonchandler changed the base branch from main to 0.4.0dev July 25, 2023 17:30

talonchandler added 11 commits July 25, 2023 10:35

fix input channels in tests

b85ebba

black format tests

1e5639c

5 channel pol + BF test input

74160c1

test on all PRs (#383)

9734847

update waveorder dependency

ecd3ebe

support additional channels in input dataset

f6a22d8

Update docs to example settings

623a499

convert from np.uint16 to np.int32 then torch.float32

2e974a3

np.uint16 to mimic MM datasets

9fdb6b7

formatting fix

84b7480

make stack traces visible when tests fail

8fdf5d0

talonchandler marked this pull request as ready for review July 25, 2023 19:03

talonchandler requested a review from ziw-liu July 25, 2023 19:04

replace string concatenation with list of args

85b8285

ziw-liu approved these changes Jul 25, 2023

View reviewed changes

talonchandler merged commit 6e49667 into 0.4.0dev Jul 25, 2023

talonchandler deleted the fix-0.4.0dev-tests branch July 25, 2023 22:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix 0.4.0dev tests #384

Fix 0.4.0dev tests #384

talonchandler commented Jul 25, 2023 •

edited

Loading

codecov bot commented Jul 25, 2023 •

edited

Loading

talonchandler commented Jul 25, 2023 •

edited

Loading

ziw-liu commented Jul 25, 2023 •

edited

Loading

ziw-liu commented Jul 25, 2023

ziw-liu commented Jul 25, 2023

talonchandler commented Jul 25, 2023

talonchandler commented Jul 25, 2023

Fix 0.4.0dev tests #384

Fix 0.4.0dev tests #384

Conversation

talonchandler commented Jul 25, 2023 • edited Loading

codecov bot commented Jul 25, 2023 • edited Loading

Codecov Report

talonchandler commented Jul 25, 2023 • edited Loading

ziw-liu commented Jul 25, 2023 • edited Loading

ziw-liu commented Jul 25, 2023

ziw-liu commented Jul 25, 2023

talonchandler commented Jul 25, 2023

talonchandler commented Jul 25, 2023

talonchandler commented Jul 25, 2023 •

edited

Loading

codecov bot commented Jul 25, 2023 •

edited

Loading

talonchandler commented Jul 25, 2023 •

edited

Loading

ziw-liu commented Jul 25, 2023 •

edited

Loading