Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Move file stageing #10

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

Move file stageing #10

wants to merge 8 commits into from

Conversation

jedwards4b
Copy link
Contributor

Move file staging from buildlib to buildnml.
Tested using ERR_Ld5.ne30pg3_t232_wg37.1850_CAM70%LT_CLM60%BGC-CROP_CICE_MOM6_MOSART_DGLC%NOEVOLVE_WW3.derecho_intel

Fixes #9

Mvertens would like to review @alperaltuntas can you add her?

alperaltuntas
alperaltuntas reviewed Oct 29, 2024
View reviewed changes
Copy link
Member

@alperaltuntas alperaltuntas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ww3_grid executable gets built at the buildlib stage. If file staging gets moved from buildlib to buildnml and if the user attempts to, say, run ./preview_namelists before running ./case.build this change will lead to a failure due to the absence of ww3_grid executable, I think.

@jedwards4b
Copy link
Contributor Author

See buildnml line 222. Running buildnml before buildlib will print a warning about the missing executable but will no longer fail.

@alperaltuntas
Copy link
Member

I am confused about the "resolve git issue" commit that was just pushed. It appears to change the WW3 submodule hash (why?) and when I click on the new submodule, GitHub displays: "This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository."

@jedwards4b
Copy link
Contributor Author

I tried to undo that but had the same problem. I don't think that it matters because the submodule will need to be updated again when this is merged.

@alperaltuntas
Copy link
Member

The issue persists. When I create a test from scratch, buildnml doesn't generate or copy the ww3.mod_def file, causing the run to fail.

alperaltuntas
alperaltuntas requested changes Nov 7, 2024
View reviewed changes
cime_config/buildnml Outdated
safe_copy(os.path.join(input_dir, filename), os.path.join(output_dir, filename))

# Create mod_def file using ww3_grid and the grid_input files
run_cmd("ww3_grid", from_dir=output_dir)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Modifying this line as follows should fix this PR:

run_cmd("./ww3_grid > mod_def.ww3.log", from_dir=output_dir)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better yet, I suggest replacing lines 218-223 with the following:

        if os.path.isfile(os.path.join(output_dir,"ww3_grid")): 
            run_cmd("./ww3_grid > mod_def.ww3.log", from_dir=output_dir)
            if not os.path.isfile(os.path.join(output_dir,"mod_def.ww3")):
                raise RuntimeError("mod_def.ww3 was not created, check mod_def.ww3.log for errors.")
            shutil.move(os.path.join(output_dir,"mod_def.ww3"), os.path.join(rundir, "mod_def.ww3"))
        else:
            logger.warning("ww3_grid file not found. The mod_def.ww3 file will be created after the build phase.")

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually, even these changes don't fix the issue with TEST runs. It appears like, when you run a TEST case, the buildnml script doesn't get called in between build and run phases. And so the mod_def file never gets created in a TEST run.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have found the toggle to skip the pnl in tests and am testing a change to unset this when ww3 is in the compset.

@jedwards4b
Copy link
Contributor Author

I was able to reproduce the issue, I'll work on it this morning.

@jedwards4b
Copy link
Contributor Author

I've added a fix to my cime branch in ESMCI/cime#4689
That branch will require other components to be updated as well, so here is the specific change that you can try in your current sandbox:

diff --git a/CIME/SystemTests/system_tests_common.py b/CIME/SystemTests/system_tests_common.py
index 70091aa29..a86483ffd 100644
--- a/CIME/SystemTests/system_tests_common.py
+++ b/CIME/SystemTests/system_tests_common.py
@@ -399,7 +399,13 @@ class SystemTestsCommon(object):
         """
         success = True
         start_time = time.time()
-        self._skip_pnl = skip_pnl
+        wav_comp = self._case.get_value("COMP_WAV")
+        # WW3 requires pnl to be run again after the build phase.
+        if wav_comp and wav_comp == "ww3":
+            self._skip_pnl = False
+        else:
+            self._skip_pnl = skip_pnl
+        
         try:
             self._resetup_case(RUN_PHASE)
             do_baseline_ops = True

@alperaltuntas
Copy link
Member

Just confirming that this CIME mod, along with the changes I suggest above, fixed the issue.

@jedwards4b
Copy link
Contributor Author

I've added you to my fork, please feel free to push your additional changes.

@alperaltuntas
Copy link
Member

Done.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alperaltuntas alperaltuntas alperaltuntas approved these changes

@mvertens mvertens Awaiting requested review from mvertens

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Code in buildlib that needs to be in buildnml
2 participants
@jedwards4b @alperaltuntas