mARTin-B78/DGX-Spark-ComfyUI
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Update CUDA_MODULE_LOADING to use LAZY

Browse Source 
Changed CUDA module loading strategy from EAGER to LAZY for improved model loading speed.
This commit is contained in:
luix93 2026-03-28 16:51:43 +01:00 committed by GitHub
parent 0b07cb4a7e
commit 7b2f184517
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docker-compose.yml
View File

@ -42,7 +42,7 @@ services:
PYTORCH_NO_CUDA_MEMORY_CACHING: "1" PYTORCH_NO_CUDA_MEMORY_CACHING: "1"
CUDA_DEVICE_MAX_CONNECTIONS: "1" CUDA_DEVICE_MAX_CONNECTIONS: "1"
CUDA_DEVICE_MAX_COPY_CONNECTIONS: "4" CUDA_DEVICE_MAX_COPY_CONNECTIONS: "4"
CUDA_MODULE_LOADING: "CUDA" # EAGER works as well, however CUDA loads models in memory a lot faster CUDA_MODULE_LOADING: "LAZY" # EAGER works as well, however LAZY loads models in memory a lot faster
CUDA_MANAGED_FORCE_DEVICE_ALLOC: "1" CUDA_MANAGED_FORCE_DEVICE_ALLOC: "1"
OMP_NUM_THREADS: "20" OMP_NUM_THREADS: "20"
CUBLAS_WORKSPACE_CONFIG: ":0:0" CUBLAS_WORKSPACE_CONFIG: ":0:0"