measureIQ for N&S is broken in 3.1.x

[KathleenLabrie]

Branch: release/3.1.x

measureIQ for N&S issues warning and cannot find any good sources. Used to work. Not urgent, I'm not holding 3.1 for this, but it should be looked at at some point.

================================================================================
RECIPE: measureIQ
================================================================================
   PRIMITIVE: measureIQ
   --------------------
      PRIMITIVE: tileArrays
      ---------------------
      WARNING - N20080112S0398_skyCorrected.fits has nothing to tile, as tile_all=False but each array has only one amplifier.
      .
/home/pipetest/testconda/susd3.9_20220816-test/lib/python3.9/site-packages/numpy/lib/function_base.py:4527: RuntimeWarning: invalid value encountered in subtract
  diff_b_a = subtract(b, a)
/home/pipetest/testconda/susd3.9_20220816-test/lib/python3.9/site-packages/numpy/lib/function_base.py:4527: RuntimeWarning: invalid value encountered in subtract
  diff_b_a = subtract(b, a)
/home/pipetest/testconda/susd3.9_20220816-test/lib/python3.9/site-packages/numpy/lib/function_base.py:4527: RuntimeWarning: invalid value encountered in subtract
  diff_b_a = subtract(b, a)
   WARNING - No good sources found in N20080112S0398_skyCorrected.fits
   .
	Wrote N20080112S0398_iqMeasured.fits in output directory

reduce completed successfully.

To reproduce:
reduce -r measureIQ N20080112S0398_skyCorrected.fits

The file N20080112S0398_skyCorrected.fits can be found here: https://drive.google.com/drive/folders/1eJgTxJN0FwmW8zF-YTWztSuny7pmV2iu?usp=share_link

The input was produce with this recipe:

    p.prepare()
    p.addDQ()
    p.addVAR(read_noise=True)
    p.overscanCorrect()
    p.ADUToElectrons()
    p.addVAR(poisson_noise=True)
    p.findAcquisitionSlits()
    p.skyCorrectNodAndShuffle()

[MichaelRFairhurst]

Hello Kathleen & team.

I bisected this to be introduced in 3591f83

[MichaelRFairhurst]

This might not be a bug, and/or may just need better error messaging.

I don't have access to the drive file; to get the N&S data in attempt to repro the problem I downloaded N20080104S0206 from the archives, ran the provided recipe step by step over CLI.

If my bisect is correct, it appears that the issue is this line in fit_continuum():

                data, mask, var = NDStacker.mean(ndd)
                if mask is not None:
                    mask = (mask == 0)
                    if mask.sum() < MIN_APERTURE_WIDTH:
                        continue

The data is failing this MIN_APERTURE_WIDTH test with a value of 9 vs the necessary 10. So this may be correct/intended behavior?

MeasureIQ worked successfully on N20080104S0206:

      Filename: N20080104S0206_skyCorrected.fits
      3 source(s) used to measure IQ
      ----------------------------------------------------------
      FWHM measurement:                   1.200 +/- 0.000 arcsec
      Zenith-corrected FWHM (AM 1.17):    1.093 +/- 0.000 arcsec
      IQ range for OG515-band:              IQAny (>1.05 arcsec)
      (Requested IQ could not be determined)                          
      ----------------------------------------------------------
   
   .
	Wrote N20080104S0206_iqMeasured.fits in output directory

If it would be helpful to add a warning along the lines of "All apertures excluded due to sufficient size," and/or make the parameter MIN_APERTURE_WIDTH configurable, I would be happy to try my hand at it.

Hopefully this helps!

[KathleenLabrie]

Thanks Michael. This file used to work in a previous version of DRAGONS. Thanks for figuring out where it fails now. We'll have to understand what changed but knowing where it fails certainly helps a lot.