Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MPI Sync timeout? #143

Open
2 tasks
FG-TUM opened this issue Jul 13, 2022 · 0 comments
Open
2 tasks

MPI Sync timeout? #143

FG-TUM opened this issue Jul 13, 2022 · 0 comments
Labels
Bug Something isn't working

Comments

@FG-TUM
Copy link
Collaborator

FG-TUM commented Jul 13, 2022

Issue

The problem

The timeout mechanic could potentially cause a deadlock with MPI:

if (secondsSinceStart > timeout) {
// FIXME: MPI SYNC THIS?
SPDLOG_LOGGER_INFO(logger.get(),
"Simulation timeout hit! Time since simulation start ({} s) > Timeout ({} s))",
secondsSinceStart,
timeout);
// set the config to the number of completed iterations (hence no +/-1) for the timer calculations.
config.setValue("sim/iterations", iteration);
// abort the loop by increasing the loop counter. This also leads to triggering the visualization
iteration = lastIteration;
}

Expected behavior

No Deadlock

What needs to be done

  • Testing
  • Synchronization if necessary

How it can be tested or reproduced

Run a simulation that hits a timeout.

@FG-TUM FG-TUM added the Bug Something isn't working label Jul 13, 2022
@FG-TUM FG-TUM mentioned this issue Jul 13, 2022
30 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant