Multiple tests types using same input file

[jessecarterMOOSE]

Check these boxes if you have followed the posting rules.

• Q&A General is the most appropriate section for my question
• I have consulted the posting Guidelines on the Discussions front page
• I have searched the Discussions forum and my question has not been asked before
• I have searched the MOOSE website and the documentation does not answer my question
• I have formatted my post following the posting guidelines (screenshots as a last resort, triple back quotes around pasted text)

Question

What's a good way to do multiple tests on the output from a single input file? Example: I want to do an Exodiff and CSVdiff. Or maybe 2 CSVdiffs at different time steps. The only way I know is to put multiple tests in the tests spec but then the same problem gets run twice which seems inefficient (and potentially problematic if running in parallel where outputs could collide).

[GiudGiud]

Hello

and potentially problematic if running in parallel where outputs could collide).

You can set prereq= in the tests spec to avoid this

I would instead use Outputs/file_base to make sure the outputs of each test are different. Then they can all run in parallel.
You add this in the test spec using cli_args = Outputs/file_base=test_1_

[jessecarterMOOSE]

@milljm I only want to run the input file once and then do lots of tests on the output.

@GiudGiud The example of doing Exodiff and CSVdiff is just an example. I have PP's and VPP's and exodus and want to check them all.

[milljm]

@milljm I only want to run the input file once and then do lots of tests on the output.

You'll still want this:

  parallel_scheduling = true

to be able to do

and then do lots of tests on the output.

[milljm]

To emphasize this behavior:

$ ./run_tests -i testing_test -j 10 -t
[8.002s]       OK tests/test_harness.a
[2.002s]       OK tests/test_harness.b
[2.002s]       OK tests/test_harness.c
[2.002s]       OK tests/test_harness.d
[2.002s]       OK tests/test_harness.e
[2.002s]       OK tests/test_harness.f
[2.003s]       OK tests/test_harness.g
[2.003s]       OK tests/test_harness.h
[2.002s]       OK tests/test_harness.i
[2.002s]       OK tests/test_harness.j
-------------------------------------------------------------------------------------------
Ran 10 tests in 26.3 seconds. Average test time 2.6 seconds, maximum test time 8.0 seconds.
10 passed, 0 skipped, 0 failed

Each test is launched in serial, resulting in 26 seconds.

But when we add parallel_scheduling = true:

@@ -1,4 +1,5 @@
 [Tests]
+  parallel_scheduling = true
   [a]
     type = RunCommand
     command = "sleep 8"

$ ./run_tests -i testing_test -j 10 -t
[8.002s]       OK tests/test_harness.a
[1.998s]       OK tests/test_harness.e
[1.998s]       OK tests/test_harness.g
[1.983s]       OK tests/test_harness.h
[1.994s]       OK tests/test_harness.d
[1.993s]       OK tests/test_harness.j
[1.998s]       OK tests/test_harness.f
[1.998s]       OK tests/test_harness.c
[2.002s]       OK tests/test_harness.i
[2.002s]       OK tests/test_harness.b
-------------------------------------------------------------------------------------------
Ran 10 tests in 10.2 seconds. Average test time 2.6 seconds, maximum test time 8.0 seconds.
10 passed, 0 skipped, 0 failed

all tests after the initial pre_preq are launched concurrently. Resulting in 10 seconds total test group completion.

[jessecarterMOOSE]

What if each of my tests is a CSVdiff? Doesn't each one re-run the problem? That's what I was trying to avoid.

[milljm]

You can use the attribute:

should_execute = false

This allows the postProcessing step to commence, without running RunApp.
See https://github.com/idaholab/moose/blob/next/test/tests/test_harness/should_execute for detailed information on how this test spec file would work.