Pre and post steps

[jsemler]

This ticket is to discuss the use cases for specifying pre/post steps which could be used to drive a workflow. These would be actions that could be run before the start of a step and after a step completes.

This would allow for things like verifying state, check-pointing, or post-processing between steps. This would also allow users to define a set of conditions for driving a workflow such as determining if a step should be rerun.

[FrankD412]

I've been thinking about this one for a few days. Here's what I've come up with.

This is going to require some standardization of return codes for pre and post scripts which then map to enumerations that Maestro expects (most pre and post scripts should probably be Python which import the enums directly). Some of the codes we'd need (not in any particular order):

PrePostCode.FAIL
PrePostCode.RESTART
PrePostCode.RERUN         # Different than RESTART, rerun the workflow from scratch.
PrePostCode.CANCEL       # This could be useful once restarting is available through Maestro

I'm thinking that scripts are going to want to be lightweight and locally executed, since I think any heavier lifting would want to be a workflow step itself (and heavier post processing could be a step with its own pre and post). Other things would be input prep in pre step and input correction in post steps -- but those would have to be light.

I've also been thinking about how to execute on it, but the option of getting down a pipeline and then flagging an ancestor as having to be rerun is also a potential option. I don't know how useful that is but I can think of that getting complicated.

These are just some thoughts for now, but I figure a back and forth can help tease out requirements.

[jsemler]

I think this is reasonable. I agree the scripts should be lightweight and run locally.

I think it would be useful to allow jumping back to a given step, though I’m not sure how that would be specified in the script. Maybe by a return code and the name of the step to rerun?

[FrankD412]

With some of the recent discussions with @jsemler, should Maestro be responsible for restarting from previously executed steps?

[doutriaux1]

@FrankD412 is this still something in the pipes? I think I could use this, for example cleanup or archiving of failed simulations.