Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactoring of in-house scheduling of computational jobs to allow scaling of simcore services and reduce load on system #6666

Open
Tracked by #1730
sanderegg opened this issue Nov 5, 2024 · 0 comments
Assignees
Labels
a:director-v2 issue related with the director-v2 service

Comments

@sanderegg
Copy link
Member

sanderegg commented Nov 5, 2024

The scheduling of tasks from the director-v2 has the following issues:

  1. on each round all the computational jobs are checked (creating a lot of network calls to the dask-schedulers),
  2. every time a new pipeline is added 1. is done again even if it was just done,
  3. every time a pipeline is stopped 1. is done again,
  4. it is not scalable on multiple director-v2 replicas,

--> A distributed lock shall be used to protect each pipeline separately (this way multiple replicas can take care of different pipelines),
after adding/stopping a pipeline only that one should be re-scheduled, not all of them

@sanderegg sanderegg self-assigned this Nov 5, 2024
@sanderegg sanderegg transferred this issue from ITISFoundation/osparc-issues Nov 5, 2024
@sanderegg sanderegg added the a:director-v2 issue related with the director-v2 service label Nov 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a:director-v2 issue related with the director-v2 service
Projects
None yet
Development

No branches or pull requests

1 participant