Add writing out of ensemble mean for soil increments.

[ClaraDraper-NOAA]

Description

The GFSV17 soil analysis requires the ensemble mean increment to be added to the deterministic member. It's simplest to write it out from the EnKF, as the associated snow mask is available. These updates write out the ensemble mean soil increment when a soil update is calculated.

Addresses issue #801

Type of change

Please delete options that are not relevant.

• [ x] New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Tested that it correctly write out the ensemble mean soil increment for paranc = T and F.

I've run the global regression tests on hera. All passed except:

The runtime for hafs_4denvar_glbens_hiproc_updat is 364.487057 seconds. This has exceeded maximum allowable threshold time of 324.213506 seconds,
resulting in Failure of timethresh2 the regression test.

The above test passed all the reproducability tests.

The regional EnKF test also passed on hera.

Checklist

• [x ] My code follows the style guidelines of this project
• [x ] I have performed a self-review of my own code
• [x ] I have commented my code, particularly in hard-to-understand areas
• [x ] New and existing tests pass with my changes
• [x ] Any dependent changes have been merged and published

[RussTreadon-NOAA]

@ClaraDraper-NOAA , who would you like me to assign as the two peer reviewers for this PR?

[ClaraDraper-NOAA]

@ClaraDraper-NOAA , who would you like me to assign as the two peer reviewers for this PR?

@jswhit and @tsga have both worked on this section of the code. Or Yanjun Gan if one of them can't.

[ClaraDraper-NOAA]

@CoryMartin-NOAA could also review this (@jswhit2 is on travel this week).

[RussTreadon-NOAA]

Step 3 of GSI wiki page GSI: How to Make Changes requires that ctests be run with results reported in either the originating issue or PR.

I can run ctests on the development WCOSS2 once it returns to service.

Can someone run ctests on at least Hera? Running on Orion and Hercules is a good idea since code can behave differently on different machines.

[ClaraDraper-NOAA]

I've run the regression tests on hera. All passed except:

The runtime for hafs_4denvar_glbens_hiproc_updat is 364.487057 seconds. This has exceeded maximum allowable threshold time of 324.213506 seconds, resulting in Failure of timethresh2 the regression test.

The above test passed all the reproducability tests.

@RussTreadon-NOAA I've run the global tests on hera (see the PR description - please let me know if I should report that differently). From the webpage you just linked to it looks like I need to run the RRFS EnKF test as well. I'll do that today. Unfortunately, I don't have access to orion or hercules at the moment.

[RussTreadon-NOAA]

Thank you @ClaraDraper-NOAA . I did not catch the Hera ctest results. My mistake. I'll run on Hercules and Orion.

[ClaraDraper-NOAA]

Thank you @ClaraDraper-NOAA . I did not catch the Hera ctest results. My mistake. I'll run on Hercules and Orion.

the regional test passed. I'll amend the statement above.

[RussTreadon-NOAA]

Hercules ctest

Install ClaraDraper-NOAA:feature/sfc_ensmean at 85d0af2 along with develop at b0e3cba on Hercules. Run ctests with the following results

Test project /work/noaa/da/rtreadon/git/gsi/pr802/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/1 Test #2: rtma .............................   Passed  1568.01 sec
2/6 Test #6: global_enkf ......................***Failed  5053.41 sec
3/6 Test #3: rrfs_3denvar_rdasens .............   Passed  5068.36 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  6978.26 sec
5/6 Test #1: global_4denvar ...................   Passed  7144.54 sec
6/6 Test #4: hafs_4denvar_glbens ..............   Passed  8536.59 sec

83% tests passed, 1 tests failed out of 6

Total Test time (real) = 8536.62 sec

The following tests FAILED:
          6 - global_enkf (Failed)

The global_enkf failure is due to

The runtime for global_enkf_loproc_updat is 134.283243 seconds.  This has exceeded maximum allowable threshold time of 133.025119 seconds, resulting in Failure timethresh of the regression test.

This is not a fatal failure. A check of the enkf.x wall times does not show any anomalous behavior

global_enkf_hiproc_contrl/stdout:The total amount of wall time                        = 111.411896
global_enkf_hiproc_updat/stdout:The total amount of wall time                        = 111.500811
global_enkf_loproc_contrl/stdout:The total amount of wall time                        = 121.588227
global_enkf_loproc_updat/stdout:The total amount of wall time                        = 122.041134

A rerun of global_enkf passed

Test project /work/noaa/da/rtreadon/git/gsi/pr802/build
    Start 6: global_enkf
1/1 Test #6: global_enkf ......................   Passed  3736.79 sec

100% tests passed, 0 tests failed out of 1

Total Test time (real) = 3737.00 sec

[RussTreadon-NOAA]

Orion ctests

Install ClaraDraper-NOAA:feature/sfc_ensmean at 0ae7c2c and develop at b0e3cba on Orion. Run ctests with the following results.

Test project /work2/noaa/da/rtreadon/git/gsi/pr802/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/6 Test #6: global_enkf ......................   Passed  2709.10 sec
2/6 Test #3: rrfs_3denvar_rdasens .............   Passed  3066.59 sec
3/6 Test #2: rtma .............................   Passed  3907.83 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  4583.26 sec
5/6 Test #4: hafs_4denvar_glbens ..............   Passed  4764.15 sec
6/6 Test #1: global_4denvar ...................   Passed  5822.31 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 5822.33 sec

[RussTreadon-NOAA]

This PR is awaiting the return of WCOSS2 to developers so WCOSS2 ctests can be run. Assuming acceptable WCOSS2 results, this PR can be merged into develop.

[RussTreadon-NOAA]

@tsga , if the changes in this PR are acceptable to you, please complete your review by marking this PR as approved. Thank you.

[tsga]

Everything looks good to me.

[RussTreadon-NOAA]

Thank you @tsga for your approval. This is odd. You should see an "Add your review" button at the top of PR #802.

1. click on the "Add your review" button
2. click on "Review changes"
3. select "Approve"
4. click on "Submit review"

You know this since you've reviewed PRs in other repos. Maybe the above buttons don't show up for external collaborators.

[tsga]

Thank you @tsga for your approval. This is odd. You should see an "Add your review" button at the top of PR #802.

1. click on the "Add your review" button
2. click on "Review changes"
3. select "Approve"
4. click on "Submit review"

You know this since you've reviewed PRs in other repos. Maybe the above buttons don't show up for external collaborators.

Thank you @RussTreadon-NOAA. They did show up once I followed the steps you outlined.

[RussTreadon-NOAA]

WCOSS2 ctests

Install ClaraDraper-NOAA:feature/sfc_ensmean @ 0ae7c2c and develop @ b0e3cba on Cactus. Run ctests with the following results

Test project /lfs/h2/emc/da/noscrub/russ.treadon/git/gsi/pr802/build
    Start 1: global_4denvar
    Start 2: rtma
    Start 3: rrfs_3denvar_rdasens
    Start 4: hafs_4denvar_glbens
    Start 5: hafs_3denvar_hybens
    Start 6: global_enkf
1/6 Test #3: rrfs_3denvar_rdasens .............   Passed  731.98 sec
2/6 Test #6: global_enkf ......................   Passed  859.25 sec
3/6 Test #2: rtma .............................   Passed  975.17 sec
4/6 Test #5: hafs_3denvar_hybens ..............   Passed  1221.59 sec
5/6 Test #4: hafs_4denvar_glbens ..............   Passed  1221.83 sec
6/6 Test #1: global_4denvar ...................   Passed  1683.65 sec

100% tests passed, 0 tests failed out of 6

Total Test time (real) = 1683.76 sec