PermissionError: [WinError 5] Access is denied if using build multithreads > 3 #342
Comments
|
Hello, take a look to #339, same problem here. |
|
Moving clcache dir out of user profile dir doesn't work for me. I moved the C:\steven\clcache-fork\pyinstaller\dist\clcache_main\ to C:\clcache_main, and the build still fail with threads > 4. Below is the failure log: [7000] Failed to execute script clcache_main |
|
you should move clcache dir, not clcache binary. It can be configured through CLCACHE_DIR environment |
|
Thank you! It works with threads > 4 after I set set CLCACHE_DIR=C:\clcache_dir. Is this issue a known limitation by design? Do we have plan to fix it? |
|
I think this might warrant an entry in the 'Known Issues' section of the Wiki. |
|
@frerich I added an idea to fix the issue in this comment #334 (comment) |
CLCACHE_DIR --> %LOCALAPPDATA%\clcache (e.g.'C:\\Users\\jshi19\\AppData\\Local\\clcache') details see: frerich#342 CLCACHE_OBJECT_CACHE_TIMEOUT_MS --> 120s details see: frerich#251
It is not unusual that we observed the atomic_write() has data race in multipul threads and can raise exception to stop build as below: PermissionError: [WinError 5] Access is denied. Even the %LOCALAPPDATA% suggestion in below links still cannot totally fix the exceptions in our CI build system frerich#334 (comment) frerich#342 So, simply retry the atomic_write() several times first and then just pass through if still fails. The save() work around looks only impact the statistics data accurate. Since The build expection hang is always show stopper issue for us with top priority, it's OK for us get harden stable compiler cache with cost of inaccurate statistics info.
|
I can confirm that the retry workaround from @shijunjing is working great. @frerich, please consider merging this one. |
Hello，
I'm enabling the clcache in the Uefi firmware MSVC build toolchain. I've enable thed ccache in our GCC/CLang toolchains and I hope the clcache can work in our MSVC toolchain as well.
I built the clcache through the pyinstaller and I directly hard code the clcache_main.exe as my firmware VS2015 build CL and build in VS2015 cmd environment. My firmware build is driven by Uefi python build script. In my testing, I found if I build with single thread or threads <= 3, the clcache can work well. However, if I build with threads > 4, the clcache will raise the PermissionError: [WinError 5] Access is denied, below is an example. The more threads I use, the easier this issue raise.
Is there any missing configure in my build steps? Any suggestion is appreciated.
[11116] Failed to execute script clcache_main
Traceback (most recent call last):
File "clcache_main.py", line 2, in
File "site-packages\clcache_main_.py", line 1589, in main
File "site-packages\clcache_main_.py", line 1616, in processCompileRequest
File "site-packages\clcache_main_.py", line 1668, in scheduleJobs
File "concurrent\futures_base.py", line 425, in result
File "concurrent\futures_base.py", line 384, in get_result
File "concurrent\futures\thread.py", line 57, in run
File "site-packages\clcache_main.py", line 1689, in processSingleSource
File "site-packages\clcache_main.py", line 1719, in processDirect
File "site-packages\clcache_main_.py", line 1507, in processCacheHit
File "site-packages\clcache_main_.py", line 749, in exit
File "site-packages\clcache_main_.py", line 659, in save
File "contextlib.py", line 119, in exit
File "site-packages\atomicwrites_init_.py", line 159, in open
File "site-packages\atomicwrites_init.py", line 190, in commit
File "site-packages\atomicwrites_init_.py", line 93, in replace_atomic
File "site-packages\atomicwrites_init_.py", line 75, in replace_atomic
File "site-packages\atomicwrites_init.py", line 70, in _handle_errors
PermissionError: [WinError 5] Access is denied.
NMAKE : fatal error U1077: 'C:\steven\clcache-fork\pyinstaller\dist\clcache_main\clcache_main.exe' : return code '0xffffffff'
Stop.
The text was updated successfully, but these errors were encountered: