Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pluto ignores the thread settings #3032

Closed
donovaly opened this issue Sep 29, 2024 · 6 comments
Closed

Pluto ignores the thread settings #3032

donovaly opened this issue Sep 29, 2024 · 6 comments

Comments

@donovaly
Copy link

donovaly commented Sep 29, 2024

I am new to Julia and to give it a try, I wanted to check its performance. I have a CPU with 12 cores and according to the docs, I can set the number of threads uses by Julia by setting the
JULIA_NUM_THREADS
variable:
https://docs.julialang.org/en/v1/manual/multi-threading/

in the Julia REPL I see via Threads.nthreads() that it uses the 12 threads I set.

However, Pluto ignores this. Whatever I compute, it only uses one CPU thread. But Pluto runs in a single Julia thread, meaning when I close the Julia REPL, Pluto is closed as well. Thus I think it is a bug that Julia's thread settings is ignored.

I use Julia 1.11RC4 under Windows 10 64bit and latest Pluto.

@donovaly
Copy link
Author

Meanwhile I tried this approach:

import Pluto; Pluto.run(threads=12)

But also with this, whatever I execute in Pluto, only runs on one thread.

@fonsp
Copy link
Owner

fonsp commented Oct 1, 2024

import Pluto; Pluto.run(threads=12)

This should work. Are you sure that only one thread is being used? Can you share a screenshot/video where you demonstrate this?

when I close the Julia REPL, Pluto is closed as well

This is the intended behaviour for any number of threads. Maybe you want to use multiprocessing, not multithreading?

@donovaly
Copy link
Author

donovaly commented Oct 1, 2024

This should work. Are you sure that only one thread is being used? Can you share a screenshot/video where you demonstrate this?

To test, I downloaded this notebook:
import Pluto; Pluto.run(threads=12)

There they do an image convolution, which is a nice test case. But no matter what I try, the convolution runs only on one 1 core of my CPU:
explorer_DvtkQwfwFa

I loaded Julia with
julia.exe -p auto -t auto
(instead of 'auto', I also used explicit numbers)
and Pluto with
import Pluto; Pluto.run(threads=12)
(I also tried other numbers)

However, the result is always that only one CPU core is used.

when I close the Julia REPL, Pluto is closed as well

This is the intended behaviour for any number of threads. Maybe you want to use multiprocessing, not multithreading?

My point here was that I set up several threads for Julia thus think Pluto uses them as well.

@donovaly
Copy link
Author

donovaly commented Oct 1, 2024

I found a solution that Pluto respects the setting of the JULIA_NUM_THREADS variable:

import Pluto; Pluto.run(;threads="$(Threads.nthreads())")

I think that this should be the standard as Pluto is a package and packages should respect the global settings of Julia. Here, the user knows why he sets a certain number of threads for Julia, therefore there is no need to change the user's decision. If the user really wants to override Julia's presets, he can already run Pluto with another number of threads.

So here my proposal to fix the bug:

  • If Pluto is started without an explicit settings for threads, then
    • check if there is a JULIA_NUM_THREADS environment variable and use its settings also for Pluto; otherwise just use one thread
  • If Pluto is started with an explicit settings for threads, use this

@fonsp
Copy link
Owner

fonsp commented Nov 13, 2024

I understand your frustration, but a lot of previous work has gone into this behaviour, it is not accidental. Try searching for "threads" in this github to learn more.

@fonsp fonsp closed this as not planned Won't fix, can't repro, duplicate, stale Nov 13, 2024
@donovaly
Copy link
Author

I am not frustrated. I simply don't understand why the user's will is not respected. it is me, the user, who sets the number of threads for Julia. Personally, I am happy with my thread settings. And all packages of Julia should respect my settings. That's all I want and I think this is sensible, especially since one can override the settings just for Pluto anytime late.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants