Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

problem of cudnn #514

Open
King-of-Koi opened this issue May 6, 2024 · 2 comments
Open

problem of cudnn #514

King-of-Koi opened this issue May 6, 2024 · 2 comments

Comments

@King-of-Koi
Copy link

Describe the bug

When I use from JDiffusion.pipelines import StableDiffusionPipeline, JDiffusion runs import jittor as jt. At this point, an error is reported informing me that "CUDA found but cudnn is not loaded". Upon inspection, I found that there is an error in compile_extern.py with ctypes.CDLL(ex_cudnn_path, dlopen_flags) when processing libcudnn_cnn_infer.so. However, when I run import jittor as jt in a new Python file, there is no error, and the input arguments to ctypes.CDLL are consistent in both files.

Full Log

[i 0507 02:02:24.409000 00 compiler.py:956] Jittor(1.3.8.5) src: c:\users\54960.conda\envs\diffusion\lib\site-packages\jittor
[i 0507 02:02:24.438000 00 compiler.py:957] cl at C:\Users\54960.cache\jittor\msvc\VC_____\bin\cl.exe(19.29.30133)
[i 0507 02:02:24.439000 00 compiler.py:958] cache_path: C:\Users\54960.cache\jittor\jt1.3.8\cl\py3.9.19\Windows-10-10.x07\11thGenIntelRCxaa\default
[i 0507 02:02:24.441000 00 install_cuda.py:93] cuda_driver_version: [12, 2, 0]
[i 0507 02:02:24.463000 00 init.py:411] Found C:\Users\54960.cache\jittor\jtcuda\cuda11.2_cudnn8_win\bin\nvcc.exe(11.2.67) at C:\Users\54960.cache\jittor\jtcuda\cuda11.2_cudnn8_win\bin\nvcc.exe.
[i 0507 02:02:24.544000 00 compiler.py:1011] cuda key:cu11.2.67
[i 0507 02:02:24.546000 00 init.py:227] Total mem: 15.84GB, using 5 procs for compiling.
[i 0507 02:02:25.711000 00 jit_compiler.cc:28] Load cc_path: C:\Users\54960.cache\jittor\msvc\VC_____\bin\cl.exe
[i 0507 02:02:25.719000 00 init.cc:62] Found cuda archs: [86,]
[w 0507 02:02:25.862000 00 compile_extern.py:203] CUDA related path found in LD_LIBRARY_PATH or PATH(['', 'C', '\Users\54960\.cache\jittor\jtcuda\cuda11.2_cudnn8_win\lib64', '', 'C', '\Users\54960\.cache\jittor\mkl\dnnl_win_2.2.0_cpu_vcomp\bin', '', 'C', '\Users\54960\.cache\jittor\mkl\dnnl_win_2.2.0_cpu_vcomp\lib', '', 'C', '\Users\54960\.cache\jittor\jt1.3.8\cl\py3.9.19\Windows-10-10.x07\11thGenIntelRCxaa\default', '', 'C', '\Users\54960\.cache\jittor\jt1.3.8\cl\py3.9.19\Windows-10-10.x07\11thGenIntelRCxaa\default\cu11.2.67', '', 'C', '\Users\54960\.cache\jittor\jtcuda\cuda11.2_cudnn8_win\bin', '', 'C', '\Users\54960\.cache\jittor\jtcuda\cuda11.2_cudnn8_win\lib\x64', '', 'C', '\Users\54960\.cache\jittor\msvc\win10_kits\lib\ucrt\x64', '', 'C', '\Users\54960\.cache\jittor\msvc\win10_kits\lib\um\x64', '', 'C', '\Users\54960\.cache\jittor\msvc\VC\lib', '', 'c', '\users\54960\.conda\envs\diffusion\libs', 'C', '\Users\54960\.cache\jittor\msvc\VC\\\\\_\bin', 'C', '\Users\54960\.conda\envs\diffusion', 'C', '\Users\54960\.conda\envs\diffusion\Library\mingw-w64\bin', 'C', '\Users\54960\.conda\envs\diffusion\Library\usr\bin', 'C', '\Users\54960\.conda\envs\diffusion\Library\bin', 'C', '\Users\54960\.conda\envs\diffusion\Scripts', 'C', '\Users\54960\.conda\envs\diffusion\bin', 'F', '\anaconda3\condabin', 'F', '\anaconda3', 'F', '\anaconda3\Library\mingw-w64\bin', 'F', '\anaconda3\Library\usr\bin', 'F', '\anaconda3\Library\bin', 'F', '\anaconda3\Scripts', 'C', '\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2\bin', 'C', '\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v12.2\libnvvp', 'C', '\Program Files\PlasticSCM5\server', 'C', '\Program Files\PlasticSCM5\client', 'C', '\Windows\system32', 'C', '\Windows', 'C', '\Windows\System32\Wbem', 'C', '\Windows\System32\WindowsPowerShell\v1.0', 'C', '\Windows\System32\OpenSSH', 'C', '\Program Files (x86)\NVIDIA Corporation\PhysX\Common', 'C', '\Program Files\NVIDIA Corporation\NVIDIA NvDLISR', 'C', '\Program Files\Graphviz\bin', 'C', '\Emgu\bin', 'C', '\Users\54960\Desktop\mingw64\bin', 'C', '\Program Files\dotnet', 'C', '\WINDOWS\system32', 'C', '\WINDOWS', 'C', '\WINDOWS\System32\Wbem', 'C', '\WINDOWS\System32\WindowsPowerShell\v1.0', 'C', '\WINDOWS\System32\OpenSSH', 'C', '\Program Files (x86)\NetSarang\Xshell 7', 'C', '\Program Files (x86)\NetSarang\Xftp 7', 'C', '\Program Files\NVIDIA Corporation\Nsight Compute 2023.2.0', 'C', '\Users\54960\AppData\Local\Programs\Python\Python37\Scripts', 'C', '\Users\54960\AppData\Local\Programs\Python\Python37', 'C', '\Users\54960\AppData\Local\Microsoft\WindowsApps', '.', 'C', '\Users\54960\AppData\Local\Programs\Microsoft VS Code\bin']), This path may cause jittor found the wrong libs, please unset LD_LIBRARY_PATH and remove cuda lib path in Path.
Or you can let jittor install cuda for you: python3.x -m jittor_utils.install_cuda
libcudnn_ops_infer.so
C:\Users\54960.cache\jittor\jtcuda\cuda11.2_cudnn8_win\bin\cudnn_ops_infer64_8.dll
libcudnn_ops_train.so
C:\Users\54960.cache\jittor\jtcuda\cuda11.2_cudnn8_win\bin\cudnn_ops_train64_8.dll
libcudnn_cnn_infer.so
C:\Users\54960.cache\jittor\jtcuda\cuda11.2_cudnn8_win\bin\cudnn_cnn_infer64_8.dll
Traceback (most recent call last):
File "C:\Users\54960.conda\envs\diffusion\lib\site-packages\jittor\compile_extern.py", line 235, in setup_cuda_extern
setup_cuda_lib(lib_name, extra_flags=link_cuda_extern)
File "C:\Users\54960.conda\envs\diffusion\lib\site-packages\jittor\compile_extern.py", line 296, in setup_cuda_lib
ctypes.CDLL(ex_cudnn_path, dlopen_flags)
File "C:\Users\54960.conda\envs\diffusion\lib\ctypes_init_.py", line 374, in init
self._handle = _dlopen(self._name, mode)
OSError: [WinError 127] �Ҳ���ָ���ij���

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "e:\code\JDiffusion-master\JDiffusion-master\examples\dreambooth\run_all.py", line 3, in
from JDiffusion.pipelines import StableDiffusionPipeline
File "e:\code\jdiffusion-master\jdiffusion-master\python\JDiffusion_init_.py", line 1, in
import jittor as jt
File "C:\Users\54960.conda\envs\diffusion\lib\site-packages\jittor_init_.py", line 25, in
from . import compile_extern
File "C:\Users\54960.conda\envs\diffusion\lib\site-packages\jittor\compile_extern.py", line 598, in
setup_cuda_extern()
File "C:\Users\54960.conda\envs\diffusion\lib\site-packages\jittor\compile_extern.py", line 247, in setup_cuda_extern
LOG.f(msg)
File "C:\Users\54960.conda\envs\diffusion\lib\site-packages\jittor_utils_init_.py", line 104, in f
def f(self, *msg): self.log('f', 0, *msg)
File "C:\Users\54960.conda\envs\diffusion\lib\site-packages\jittor_utils_init
.py", line 89, in _log
cc.log(fileline, level, verbose, msg)
RuntimeError: [f 0507 02:02:26.633000 00 compile_extern.py:247] CUDA found but cudnn is not loaded:
Develop version of CUDNN not found,
please refer to CUDA offical tar file installation:
https://docs.nvidia.com/deeplearning/cudnn/install-guide/index.html#installlinux-tar
or you can let jittor install cuda and cudnn for you:

python3.9 -m jittor_utils.install_cuda

@LDYang694
Copy link
Collaborator

As mentioned in the error message, the cudnn environment was not found on your computer. jittor provides you with the environment script python3.9 -m jittor_utils.install_cuda.

@King-of-Koi
Copy link
Author

As mentioned in the error message, the cudnn environment was not found on your computer. jittor provides you with the environment script python3.9 -m jittor_utils.install_cuda.

I have tried the command, and CUDA has been correctly installed. As I mentioned in the question, after creating a new project, importing Jittor can correctly find cuDNN, but when importing Jittor in JDiffusion, it cannot find cuDNN. I have observed the error method's input through the log, and the input parameters are the same in both instances.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants