You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The scheduling of tasks from the director-v2 has the following issues:
on each round all the computational jobs are checked (creating a lot of network calls to the dask-schedulers),
every time a new pipeline is added 1. is done again even if it was just done,
every time a pipeline is stopped 1. is done again,
it is not scalable on multiple director-v2 replicas,
--> A distributed lock shall be used to protect each pipeline separately (this way multiple replicas can take care of different pipelines),
after adding/stopping a pipeline only that one should be re-scheduled, not all of them
The text was updated successfully, but these errors were encountered:
The scheduling of tasks from the director-v2 has the following issues:
--> A distributed lock shall be used to protect each pipeline separately (this way multiple replicas can take care of different pipelines),
after adding/stopping a pipeline only that one should be re-scheduled, not all of them
The text was updated successfully, but these errors were encountered: