Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cam6_4_038: GEOS-Chem 14.4 and dependencies #1093

Merged
merged 30 commits into from
Oct 1, 2024

Conversation

lizziel
Copy link
Collaborator

@lizziel lizziel commented Jul 16, 2024

This PR updates GEOS-Chem from version 14.1.2 to 14.4.3 It also includes updating HEMCO_CESM to 2.0.0 and HEMCO to 3.9.0. In addition, there are two new submodules in CAM: (1) Cloud-J 7.7.3, now used in GEOS-Chem for computing J-values, and (2) HETP v1.1, an improved ISORROPIA for aerosol thermodynamics in GEOS-Chem.

This PR supercedes #996 which was updating to an older version of GEOS-Chem.

Closes #959

@lizziel lizziel changed the title GEOS-Chem 14.4.1 and dependencies GEOS-Chem 14.4 and dependencies Jul 23, 2024
@lkemmons
Copy link
Collaborator

ACOM (e.g., @tilmes, Doug Kinnison) think Cloud-J would be a great option to include in CESM3, as an alternative to TUV-x for now.

@lizziel lizziel marked this pull request as ready for review August 1, 2024 18:21
@lizziel
Copy link
Collaborator Author

lizziel commented Aug 1, 2024

The tests failed due to git-fleximod errors. I am looking into them now.

s                hemco hemco-cesm2_0_hemco_3_9_0 7bd8358 is out of sync with .gitmodules hemco-cesm2_0_hemco3_9_0
          e                HEMCO has no fxtag defined in .gitmodules, module at 3.9.0
                    e geos-chem-shared-docs has no fxtag defined in .gitmodules, module at cbfb5cb
            rte-rrtmgp at tag v1.7
           rrtmgp-data at tag v1.8
                  mpas at hash b8c33daa4
                 cosp2 at tag v2.1.4cesm
                 clubb at tag clubb_4ncar_20231115_540[63](https://github.com/ESCOMP/CAM/actions/runs/10201686947/job/28224132055?pr=1093#step:3:64)50
                     source_cism at tag cism_main_2.01.013
          e    CIME/non_py/cprnc has no fxtag defined in .gitmodules, module at v1.0.6
                         fox at tag 4.1.2.1
                      genf90 at hash 48169[65](https://github.com/ESCOMP/CAM/actions/runs/10201686947/job/28224132055?pr=1093#step:3:66)
                         src at tag cice6_5_0_20231221
                e              icepack has no fxtag defined in .gitmodules, module at f6ff8f7
                       fates at tag sci.1.73.0_api.35.0.0
                         fms at tag dev/ncar_0.0.1

@lizziel
Copy link
Collaborator Author

lizziel commented Aug 1, 2024

This issue is .gitmodules for cice, cime, and hemco_config is configured for regular git submodules without the additional parameters required by fleximod. I am using the cime and cice versions that are in cam_development, and the new hemco_cesm tag is built on the tag used in cam_development too. Let me know if you need me to do anything.

Copy link

@fvitt fvitt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks pretty good. I have a few questions and change requests.

test/system/TR8.sh needs to be updated to skip over the new submodules.

Have you ran the aux_cam regression test suite to ensure that these changes to mo_neu_wetdep do not affect CAMChem or WACCM?

bld/configure Outdated Show resolved Hide resolved
cime_config/buildnml Outdated Show resolved Hide resolved
bld/namelist_files/namelist_defaults_cam.xml Show resolved Hide resolved
src/chemistry/geoschem/chemistry.F90 Outdated Show resolved Hide resolved
src/chemistry/mozart/mo_neu_wetdep.F90 Show resolved Hide resolved
@fvitt
Copy link

fvitt commented Aug 8, 2024

Suggested changes to test/system/TR8.sh

diff --git a/test/system/TR8.sh b/test/system/TR8.sh
index 498c9bb5..cbdb4004 100755
--- a/test/system/TR8.sh
+++ b/test/system/TR8.sh
@@ -58,12 +58,12 @@ fi
 #Check Chemistry
 if [ -d "${CAM_ROOT}/components/cam" ]; then
 
-ruby $ADDREALKIND_EXE -r r8 -l 1 -d $CAM_ROOT/components/cam/src/chemistry -s geoschem
+ruby $ADDREALKIND_EXE -r r8 -l 1 -d $CAM_ROOT/components/cam/src/chemistry -s geoschem/geoschem_src,cloud_j,hetp
 rc=`expr $? + $rc`
 
 else
 
-ruby $ADDREALKIND_EXE -r r8 -l 1 -d $CAM_ROOT/src/chemistry -s geoschem
+ruby $ADDREALKIND_EXE -r r8 -l 1 -d $CAM_ROOT/src/chemistry -s geoschem/geoschem_src,cloud_j,hetp
 rc=`expr $? + $rc`
 
 fi

@lizziel
Copy link
Collaborator Author

lizziel commented Aug 9, 2024

When I run submit aux_cam test suite I get a lot of errors because I am on derecho instead of izumi. Is it still useful for me to test with it?
Update: it is running derecho tests instead

@fvitt
Copy link

fvitt commented Aug 9, 2024

When I run submit aux_cam test suite I get a lot of errors because I am on derecho instead of izumi. Is it still useful for me to test with it? Update: it is running derecho tests instead

test_driver.sh in test/system/ will launch the aux_cam tests on derecho and izumi.

This is how I submit the tests on derecho:

env BL_TESTDIR=/glade/campaign/cesm/community/amwg/cam_baselines/cam6_4_018_intel test_driver.sh -f

@lizziel
Copy link
Collaborator Author

lizziel commented Aug 9, 2024

I now have lots of tests in ~elundgren/scratch. Could you tell me what tests to look at? Or do you run a script to evaluate all of them at once?

@fvitt
Copy link

fvitt commented Aug 9, 2024

I now have lots of tests in ~elundgren/scratch. Could you tell me what tests to look at? Or do you run a script to evaluate all of them at once?

To get a summary of the test results try thist:

cs.status.20240809_111703_3ffrar -s

I see that you have a izumi tests as well as derecho tests. You must be launching the tests differntly.

When I use the test_driver.sh script it creates an aux_cam sub dir under my scratch. For an example, see

/glade/derecho/scratch/fvitt/aux_cam_20240807162715

@lizziel
Copy link
Collaborator Author

lizziel commented Aug 14, 2024

@fvitt, here are the aux_cam test results. Let me know if we should be concerned about any of these. I am not sure why some are still pending given this was done last week.

  PASS ERC_D_Ln9.f19_f19_mg17.QPC6.derecho_intel.cam-outfrq3s_cosp
  PASS ERC_D_Ln9.f19_f19_mg17.QPMOZ.derecho_intel.cam-outfrq3s
  PASS ERC_D_Ln9.f19_f19_mg17.QPX2000.derecho_intel.cam-outfrq3s
  PASS ERC_D_Ln9.ne16_ne16_mg17.FADIAB.derecho_intel.cam-terminator
  PASS ERC_D_Ln9.ne16_ne16_mg17.QPC5HIST.derecho_intel.cam-outfrq3s_usecase
  PASS ERC_D_Ln9_P144x1.ne16pg3_ne16pg3_mg17.QPC6HIST.derecho_intel.cam-outfrq3s_ttrac_usecase
  PASS ERC_D_Ln9.T42_T42_mg17.FDABIP04.derecho_intel.cam-outfrq3s_usecase
  PASS ERC_D_Ln9.T42_T42_mg17.FHS94.derecho_intel.cam-outfrq3s_usecase
  PASS ERI_D_Ln18.f45_f45_mg37.QPC41850.derecho_intel.cam-co2rmp_usecase
  PASS ERP_D_Ln9.f19_f19_mg17.QPC6.derecho_intel.cam-outfrq9s
  PASS ERP_D_Ln9.ne30pg3_ne30pg3_mg17.FLTHIST.derecho_intel.cam-outfrq9s
  PASS ERP_D_Ln9.ne30pg3_ne30pg3_mg17.FLTHIST.derecho_intel.cam-outfrq9s_rrtmgp
  PASS ERP_D_Ln9_P64x2.f09_f09_mg17.QSC6.derecho_intel.cam-outfrq9s
  PEND ERP_Ld3.f09_f09_mg17.FWHIST.derecho_intel.cam-reduced_hist1d
  PEND ERP_Lh12.f19_f19_mg17.FW4madSD.derecho_intel.cam-outfrq3h
  PASS ERP_Ln9.C96_C96_mg17.F2000climo.derecho_intel.cam-outfrq9s_mg3
  PASS ERP_Ln9.f09_f09_mg17.F1850.derecho_intel.cam-outfrq9s
  PASS ERP_Ln9.f09_f09_mg17.F2000climo.derecho_intel.cam-outfrq9s
  PASS ERP_Ln9.f09_f09_mg17.F2010climo.derecho_intel.cam-outfrq9s
  PEND ERP_Ln9.f09_f09_mg17.FCSD_HCO.derecho_intel.cam-outfrq9s
  PASS ERP_Ln9.f09_f09_mg17.FHIST_BDRD.derecho_intel.cam-outfrq9s
  PEND ERP_Ln9.f19_f19_mg17.FWsc1850.derecho_intel.cam-outfrq9s
  PEND ERP_Ln9.ne30pg3_ne30pg3_mg17.FCnudged.derecho_intel.cam-outfrq9s
  PEND ERP_Ln9.ne30pg3_ne30pg3_mg17.FW2000climo.derecho_intel.cam-outfrq9s
  PASS ERP_Ln9_P24x3.f45_f45_mg37.QPWmaC6.derecho_intel.cam-outfrq9s_mee_fluxes
  PEND ERS_Ld3.f10_f10_mg37.F1850.derecho_intel.cam-outfrq1d_14dec_ghg_cam7
  PEND ERS_Ln9.f09_f09_mg17.FX2000.derecho_intel.cam-outfrq9s
  PASS ERS_Ln9.f19_f19_mg17.FSPCAMS.derecho_intel.cam-outfrq9s
  PEND ERS_Ln9.f19_f19_mg17.FXSD.derecho_intel.cam-outfrq9s
  PASS ERS_Ln9.ne0TESTONLYne5x4_ne0TESTONLYne5x4_mg37.FADIAB.derecho_intel.cam-outfrq3s_refined
  PASS ERS_Ln9_P288x1.mpasa120_mpasa120.F2000climo.derecho_intel.cam-outfrq9s_mpasa120
  PASS ERS_Ln9_P36x1.mpasa480_mpasa480.F2000climo.derecho_intel.cam-outfrq9s_mpasa480
  PASS SCT_D_Ln7.T42_T42_mg17.QPC5.derecho_intel.cam-scm_prep
  PASS SMS_D_Ld2.f19_f19_mg17.QPC5HIST.derecho_intel.cam-volc_usecase
  PASS SMS_D_Ld5.f19_f19_mg17.PC4.derecho_intel.cam-cam4_port5d
  PASS SMS_D_Ln9.f09_f09_mg17.FCts2nudged.derecho_intel.cam-outfrq9s_leapday
  PASS SMS_D_Ln9.f09_f09_mg17.FCvbsxHIST.derecho_intel.cam-outfrq9s
  PASS SMS_D_Ln9.f09_f09_mg17.FSD.derecho_intel.cam-outfrq9s
  PASS SMS_D_Ln9.f19_f19_mg17.FWma2000climo.derecho_intel.cam-outfrq9s
  PASS SMS_D_Ln9.f19_f19_mg17.FWma2000climo.derecho_intel.cam-outfrq9s_waccm_ma_mam4
  PASS SMS_D_Ln9.f19_f19_mg17.FXHIST.derecho_intel.cam-outfrq9s_amie
  PASS SMS_D_Ln9.f19_f19_mg17.QPC2000climo.derecho_intel.cam-outfrq3s_usecase
  PASS SMS_D_Ln9.f19_f19_mg17.QPC5M7.derecho_intel.cam-outfrq9s
  PASS SMS_D_Ln9.ne16_ne16_mg17.QPX2000.derecho_intel.cam-outfrq9s
  PASS SMS_D_Ln9.ne16pg3_ne16pg3_mg17.FX2000.derecho_intel.cam-outfrq9s
  PASS SMS_D_Ln9.ne30pg3_ne30pg3_mg17.FCts4MTHIST.derecho_intel.cam-outfrq9s
  PASS SMS_D_Ln9.ne30pg3_ne30pg3_mg17.FMTHIST.derecho_intel.cam-outfrq9s
  FAIL SMS_D_Ln9_P1280x1.ne0ARCTICne30x4_ne0ARCTICne30x4_mt12.FHIST.derecho_intel.cam-outfrq9s
  FAIL SMS_D_Ln9_P1280x1.ne0CONUSne30x8_ne0CONUSne30x8_mt12.FCHIST.derecho_intel.cam-outfrq9s
  PASS SMS_D_Ln9_P1280x1.ne30pg3_ne30pg3_mg17.FCLTHIST.derecho_intel.cam-outfrq9s
  FAIL SMS_D_Ln9.T42_T42.FSCAM.derecho_intel.cam-outfrq9s
  PASS SMS_Ld1.f09_f09_mg17.FCHIST_GC.derecho_intel.cam-outfrq1d
  PEND SMS_Ld1.f09_f09_mg17.FW2000climo.derecho_intel.cam-outfrq1d
  PEND SMS_Ld1.ne30pg3_ne30pg3_mg17.FC2010climo.derecho_intel.cam-outfrq1d
  PASS SMS_Ld5.f09_f09_mg17.PC6.derecho_intel.cam-cam6_port_f09
  PEND SMS_Lh12.f09_f09_mg17.FCSD_HCO.derecho_intel.cam-outfrq3h
  PASS SMS_Lm13.f10_f10_mg37.F2000climo.derecho_intel.cam-outfrq1m
  PASS SMS_Ln9.f09_f09_mg17.F2010climo.derecho_intel.cam-nudging
  PEND SMS_Ln9.f09_f09_mg17.FW1850.derecho_intel.cam-reduced_hist3s
  PASS SMS_Ln9.f19_f19.F2000climo.derecho_intel.cam-silhs
  PASS SMS_Ln9.f19_f19_mg17.FHIST.derecho_intel.cam-outfrq9s_nochem
  PEND SMS_Ln9.ne30pg3_ne30pg3_mg17.FW2000climo.derecho_intel.cam-outfrq9s_rrtmgp

@lizziel
Copy link
Collaborator Author

lizziel commented Aug 14, 2024

When I run submit aux_cam test suite I get a lot of errors because I am on derecho instead of izumi. Is it still useful for me to test with it? Update: it is running derecho tests instead

test_driver.sh in test/system/ will launch the aux_cam tests on derecho and izumi.

This is how I submit the tests on derecho:

env BL_TESTDIR=/glade/campaign/cesm/community/amwg/cam_baselines/cam6_4_018_intel test_driver.sh -f

Good to know. I submitted via this since it is what I use for geoschem category tests:
qcmd -A UHAR0022 ./create_test --xml-category aux_cam -p UHAR0022

@cacraigucar
Copy link
Collaborator

When I run submit aux_cam test suite I get a lot of errors because I am on derecho instead of izumi. Is it still useful for me to test with it? Update: it is running derecho tests instead

test_driver.sh in test/system/ will launch the aux_cam tests on derecho and izumi.
This is how I submit the tests on derecho:

`env BL_TESTDIR=/glade/campaign/cesm/community/amwg/cam_baselines/cam6_4_018_intel test_driver.sh -f`

Good to know. I submitted via this since it is what I use for geoschem category tests: qcmd -A UHAR0022 ./create_test --xml-category aux_cam -p UHAR0022

Using the example that Francis provided, you can add a flag --cesm geoschem and that will run just the geoschem runs.

In summary:
env BL_TESTDIR=/glade/campaign/cesm/community/amwg/cam_baselines/cam6_4_018_intel test_driver.sh -f will run aux_cam jobs

env BL_TESTDIR=/glade/campaign/cesm/community/amwg/cam_baselines/cam6_4_018_intel test_driver.sh -f --cesm geoschem will run geoschem jobs

The test_driver.sh script is how all AMP SEs run our regression testing.

@lizziel
Copy link
Collaborator Author

lizziel commented Aug 29, 2024

Thanks for the tips @cacraigucar. @fvitt, could you give me an update on where this PR stands. I see there are now conflicts due to new commits in cam_development. I can rebase and test again next week. Otherwise there are no remaining updates to add since I believe I addressed all of your comments. I am not sure about the test results above for non-GEOS-Chem runs and will need help deciphering those.

@lizziel
Copy link
Collaborator Author

lizziel commented Sep 3, 2024

@fvitt, I have rebased locally and will run tests on derecho and izumi prior to pushing. However, when I try to use test_driver.sh I get this error:
ERROR: Must set the environment variable CAM_ACCOUNT

Should I set this to the GEOS-Chem project or to some other account?

@fvitt
Copy link

fvitt commented Sep 3, 2024

@fvitt, I have rebased locally and will run tests on derecho and izumi prior to pushing. However, when I try to use test_driver.sh I get this error: ERROR: Must set the environment variable CAM_ACCOUNT

Should I set this to the GEOS-Chem project or to some other account?

Yes, use the GEOS-Chem project number.

@lizziel
Copy link
Collaborator Author

lizziel commented Sep 3, 2024

@fvitt, now I am getting ERROR: Must specify CAM_FC. Do you have a sample environment file with env settings I should use with test_driver.sh? These are the ones I need based on the file:

  echo "Supported ENVIRONMENT variables"
  echo "BL_TESTDIR:        Default = none (used to set baseline compare dir)"
  echo "CAM_ACCOUNT:       Default = none"
  echo "CAM_BATCHQ:        Default = machine dependent"
  echo "CAM_FC:            Default = machine dependent"
  echo "CAM_INPUT_TESTS:   Default = tests_pretag_<machine>[_<compiler>]"
  echo "CAM_RESTART_TASKS: Default = 64"
  echo "CAM_RETAIN_FILES:  Default = FALSE"
  echo "CAM_ROOT:          Default = set relative to CAM_SCRIPTDIR"
  echo "NB: If script is not called as ./`basename ${0}`, CAM_ROOT must be specified"
  echo "CAM_SCRIPTDIR:     Default = <current directory>"
  echo "CAM_TAG:           Default = none (used to set CESM baseline dir)"
  echo "CAM_TASKS:         Default = (depends on system)"
  echo "CAM_TESTDIR:       Default = <user_scratch_dir>/test-driver.<jobid>"
  echo ""
  echo "Less common ENVIRONMENT variables"
  echo "CALDERA_BATCHQ:    Default = caldera"
  echo "CAM_RBOPTIONS:     Default = build_only"
  echo "CAM_SOFF:          Default = none (stop of first test fail if TRUE)"
  echo "CIME_MODEL:        Default = none (should be set to cesm)"
  echo "EMAIL:             Default = [email protected]"

@fvitt
Copy link

fvitt commented Sep 3, 2024

@lizziel
Try submitting the tests on izumi like this:

env CAM_FC=NAG BL_TESTDIR=/fs/cgd/csm/models/atm/cam/pretag_bl/cam6_4_027_nag test_driver.sh -f
env CAM_FC=GNU BL_TESTDIR=/fs/cgd/csm/models/atm/cam/pretag_bl/cam6_4_027_gnu test_driver.sh -f

@lizziel
Copy link
Collaborator Author

lizziel commented Sep 3, 2024

Still not working...

$ env CAM_FC=NAG BL_TESTDIR=/fs/cgd/csm/models/atm/cam/pretag_bl/cam6_4_027_nag ./test_driver.sh -f --cesm geoschem

CESM_BASELINE /fs/cgd/csm/models/atm/cam/pretag_bl/cam6_4_027_nag not found.  Check BL_TESTDIR for correct tag name.

@fvitt
Copy link

fvitt commented Sep 3, 2024

Still not working...

$ env CAM_FC=NAG BL_TESTDIR=/fs/cgd/csm/models/atm/cam/pretag_bl/cam6_4_027_nag ./test_driver.sh -f --cesm geoschem

CESM_BASELINE /fs/cgd/csm/models/atm/cam/pretag_bl/cam6_4_027_nag not found.  Check BL_TESTDIR for correct tag name.

Did you try this on izumi?

@lizziel
Copy link
Collaborator Author

lizziel commented Sep 3, 2024

Ah, I did not notice you switched to izumi. I am trying to run tests using test_driver.sh on derecho prior to pushing. I do not have the code on izumi right now.

lizziel and others added 18 commits October 1, 2024 10:38
- Update GEOS-Chem and dependencies in .gitmodules
- Remove obsolete Externals files

Signed-off-by: Lizzie Lundgren <[email protected]>

Update changelog

Update GEOS-Chem to version 14.4.2 release tag

Signed-off-by: Lizzie Lundgren <[email protected]>
This commit also moves the location of the GEOS-Chem compsets within
the config_compsets.xml file to be by their corresponding cam-chem
compset.

Signed-off-by: Lizzie Lundgren <[email protected]>
Signed-off-by: Lizzie Lundgren <[email protected]>
Signed-off-by: Lizzie Lundgren <[email protected]>
… def

Including %NUDG after %HEMCO rather than before it results in the omission
of the nudging_nl namelist in atm_in.

Signed-off-by: Lizzie Lundgren <[email protected]>
Signed-off-by: Lizzie Lundgren <[email protected]>
Updates for compatibility with GEOS-Chem are now merged into HETP and
released as tag HETPv1.1. This removes the prior need to use the
fork in the GEOS-Chem github organization. This is a zero diff update.

Signed-off-by: Lizzie Lundgren <[email protected]>
Signed-off-by: Lizzie Lundgren <[email protected]>
This update is a zero diff update. The url may change again in the
future once an institutional github is available for Environment and
Climate Change Canada. In the meantime the GEOS-Chem fork will be used.

Signed-off-by: Lizzie Lundgren <[email protected]>
Signed-off-by: Lizzie Lundgren <[email protected]>
@lizziel
Copy link
Collaborator Author

lizziel commented Oct 1, 2024

This PR is now rebased on cam6_4_037 and the changelog is updated.

@fvitt
Copy link

fvitt commented Oct 1, 2024

@lizziel
I have submitted the regression tests. I assume we are expecting bit-for-bit unchanged results for all configurations that are not GEOSCHEM.

@cacraigucar cacraigucar changed the title cam6_4_038?: GEOS-Chem 14.4 and dependencies cam6_4_038: GEOS-Chem 14.4 and dependencies Oct 1, 2024
@lizziel
Copy link
Collaborator Author

lizziel commented Oct 1, 2024

@fvitt, that is right.

@lizziel
Copy link
Collaborator Author

lizziel commented Oct 1, 2024

Although @jimmielin told me occasionally FCSD_HCO has differences. If you see this then it is safe to say it is a pre-existing issue.

@jimmielin
Copy link
Member

Hi @lizziel @fvitt, I think FCSD_HCO is expected to have changes as well due to the updated HEMCO_CESM and HEMCO externals (which address some, but not all issues with restarts and b4b)

@fvitt fvitt merged commit 1abe4a8 into ESCOMP:cam_development Oct 1, 2024
2 checks passed
gold2718 pushed a commit to gold2718/CAM that referenced this pull request Oct 16, 2024
Merge pull request ESCOMP#1093 from geoschem/feature/geoschem_14.4

cam6_4_038: GEOS-Chem 14.4 and dependencies

ESCOMP commit: 1abe4a8
gold2718 pushed a commit to gold2718/CAM that referenced this pull request Oct 16, 2024
Merge pull request ESCOMP#1093 from geoschem/feature/geoschem_14.4

cam6_4_038: GEOS-Chem 14.4 and dependencies

ESCOMP commit: 1abe4a8
gold2718 pushed a commit to gold2718/CAM that referenced this pull request Nov 8, 2024
Merge pull request ESCOMP#1093 from geoschem/feature/geoschem_14.4

cam6_4_038: GEOS-Chem 14.4 and dependencies

ESCOMP commit: 1abe4a8
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Tag
Development

Successfully merging this pull request may close these issues.

5 participants