Skip to content

Commit

Permalink
tests: add example feature files
Browse files Browse the repository at this point in the history
  • Loading branch information
giuseppe-steduto committed Sep 5, 2023
1 parent ca257a8 commit 395dbba
Show file tree
Hide file tree
Showing 3 changed files with 81 additions and 0 deletions.
38 changes: 38 additions & 0 deletions tests/log-messages.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@

# Tests for the expected workflow log messages

Feature: Log messages

As a researcher,
I want to be able to see the log messages of my workflow execution,
So that I can verify that the workflow ran correctly.

Scenario: The workflow start has produced the expected messages
When the workflow is finished
Then the engine logs should contain "yadage.creators | MainThread | INFO | initializing workflow"
And the engine logs should contain "yadage.wflowview | MainThread | INFO | added </all_bkg_mc/0/init:0|defined|unknown>"
And the engine logs should contain "yadage.wflowview | MainThread | INFO | added </signal/0/init:0|defined|unknown>"

Scenario: The plotting step has produced the expected messages
When the workflow is finished
Then the engine logs should contain "adage.pollingexec | MainThread | INFO | submitting nodes [</plot:0|defined|known>"
And the job logs for the "plot" step should contain
"""
PARAMETER DEFINITIONS:
NO. NAME VALUE STEP SIZE LIMITS
1 Lumi 1.00000e+00 5.00000e-01 0.00000e+00 1.00000e+01
2 SigXsecOverSM 1.00000e+00 3.00000e-01 0.00000e+00 3.00000e+00
3 alpha_mc1_shape_conv 0.00000e+00 1.00000e+00 -5.00000e+00 5.00000e+00
4 alpha_mc1_weight_var1 0.00000e+00 1.00000e+00 -5.00000e+00 5.00000e+00
5 alpha_mc2_shape_conv 0.00000e+00 1.00000e+00 -5.00000e+00 5.00000e+00
6 alpha_mc2_weight_var1 0.00000e+00 1.00000e+00 -5.00000e+00 5.00000e+00
"""
And the engine logs should contain "adage.node | MainThread | INFO | node ready </plot:0|success|known>"

Scenario: The hepdata step has produced the expected messages
When the workflow is finished
Then the job logs for the "hepdata" step should contain
"""
INFO:InputArguments -- RooAbsReal::createHistogram(L_x_signal_channel1_overallSyst_x_Exp) INFO: Model has intrinsic binning definition, selecting that binning for the histogram
"""
And the engine logs should contain "adage.node | MainThread | INFO | node ready </hepdata:0|success|known>"
11 changes: 11 additions & 0 deletions tests/run-duration.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
# Tests for the workflow execution duration time

Feature: Run duration

As a researcher,
I want to verify that my workflow finishes in a reasonable amount of time,
so that I can stay assured that there are no unusual problems with computing resources.

Scenario: The workflow terminates in a reasonable amount of time
When the workflow execution completes
Then the workflow run duration should be less than 30 minutes
32 changes: 32 additions & 0 deletions tests/workspace-files.feature
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Tests for the presence of files in the workspace

Feature: Workspace files

As a researcher,
I want to make sure that my workflow produces the expected files,
and that the relevant input files are present in the workspace,
so that I can be sure that the workflow outputs are correct.

Scenario: The workspace downloads the relevant input files
When the workflow execution completes
Then the workspace should contain "data/read_0/output_one.root"
And the workspace should contain "signal/read_0/output_one.root"
And the workspace should contain "all_bkg_mc/run_mc_1/read_0/output_one.root"

Scenario: The files used to build the outputs contain the expected values
When the workflow is finished
Then the file "plot/nominal_vals.yml" should include
"""
Lumi:
err: 0.0
max: 10.0
min: 0.0
val: 1.0
"""

Scenario: The expected plots are present in the workspace
When the workflow is finished
Then the workspace should include "plot/prefit.pdf"
And the workspace should include "plot/postfit.pdf"
And all the outputs should be included in the workspace

0 comments on commit 395dbba

Please sign in to comment.