From 298e766388bd406ca01d9dcabb081716e31f1c12 Mon Sep 17 00:00:00 2001 From: luix93 <76565668+luix93@users.noreply.github.com> Date: Sat, 14 Mar 2026 16:44:03 +0100 Subject: [PATCH] Update README.md --- README.md | 177 +++++++++++++++++++++++++++++++++++++++++++++++++++++- 1 file changed, 175 insertions(+), 2 deletions(-) diff --git a/README.md b/README.md index 55f11b7..3a39a17 100644 --- a/README.md +++ b/README.md @@ -1,2 +1,175 @@ -# DGX-Spark-ComfyUI -An optimized setup for running ComfyUI on DGX Spark +# 🚀 ComfyUI on DGX Spark (Blackwell GB10) + +A Docker Compose setup for running [ComfyUI](https://github.com/comfyanonymous/ComfyUI) on the **NVIDIA DGX Spark** (Grace-Blackwell GB10), with a mobile-friendly UI included. + +Built specifically to handle the quirks of the **sm_121 / compute 12.1** architecture and its **unified CPU-GPU memory fabric**. + +--- + +## ✨ Features + +- **CUDA 13.1 base** — full `nvcc` support for GB10 (`sm_121`), enabling CUDA extension compilation +- **PyTorch cu130** — prebuilt ARM64 wheels from PyTorch's cu130 index +- **SageAttention 2** — compiled from source directly against sm_121 for full hardware attention acceleration +- **Comfy Kitchen** (`comfy_kitchen`) — NVFP4 quantization support for Blackwell +- **Unified-memory optimized flags** — carefully tuned `COMFYUI_FLAGS` that avoid fighting the Grace-Blackwell memory fabric +- **Double-VRAM bug fix** — patches `comfy/utils.py` to set `copy=False` in `tensor.to()`, fixing the double memory usage on unified memory systems with `--disable-mmap` +- **ComfyUI-Manager** — auto-installed at container startup into the mounted `custom_nodes` volume +- **ComfyUIMini** — lightweight mobile/tablet UI proxying to the ComfyUI backend (optional second service) +- **Health checks** — both services expose health check endpoints for reliable `depends_on` startup ordering +- **Persistent volumes** — models, custom nodes, outputs, inputs, user settings, and workflows are all mounted from the host + +--- + +## 🗂️ Repo Structure + +``` +. +├── Dockerfile # Main ComfyUI image (CUDA 13.1, PyTorch, SageAttention, Comfy Kitchen) +├── docker-compose.yml # Orchestrates comfyui + comfyuimini services +├── entrypoint.sh # Runtime startup: installs ComfyUI-Manager, custom node deps, launches ComfyUI +├── copy-wheels.sh # Helper to stage local .whl files into the build context +├── .env.example # Example environment file — copy to .env and customize +├── wheels/ # Place comfy_kitchen*.whl here before building +└── comfyuimini/ + └── Dockerfile # Lightweight Node.js image for ComfyUIMini mobile UI +``` + +--- + +## ⚙️ Prerequisites + +- NVIDIA DGX Spark (or any Grace-Blackwell system with `sm_121`) +- Docker with the NVIDIA Container Toolkit installed +- `comfy_kitchen-0.2.7-cp312-abi3-linux_aarch64.whl` — place in `~/comfyui/wheels/` (or wherever you keep your ML wheels) + +--- + +## 🛠️ Getting Started + +### 1. Clone the repo + +```bash +git clone https://github.com/your-username/DGX-Spark-ComfyUI.git +cd DGX-Spark-ComfyUI +``` + +### 2. Stage wheels - Not required if downloading the whole repo as the custom built comfy_kitchen wheel is included + +Copy the required wheels into the build context: + +```bash +bash copy-wheels.sh ~/comfyui/wheels +``` + +This copies `comfy_kitchen*.whl` into the local `wheels/` directory used by the Docker build. + +### 3. Configure environment + +```bash +cp .env.example .env +``` + +Edit `.env` and set your paths: + +```env +# Where your ComfyUI models live on the host +COMFYUI_HOST_PATH=/home/user/comfyui + +# Where ComfyUI data lives (custom_nodes, output, input, etc.) +COMFYUI_DATA_PATH=/home/user/comfyui +``` + +### 4. Build and start + +```bash +docker compose up --build -d +``` + +ComfyUI will be available at **`http://:8188`**. +ComfyUIMini (mobile UI) will be available at **`http://:3000`**. + +--- + +## 🔧 Optimized Flags Explained + +The default `COMFYUI_FLAGS` are tuned for the Grace-Blackwell unified memory architecture: + +| Flag | Reason | +|---|---| +| `--disable-pinned-memory` | Reduces overhead on the unified memory fabric; pinned memory is counterproductive here | +| `--use-sage-attention` | Enables SageAttention compiled for sm_121 | +| `--force-fp16` | Enables Flash Attention path in PyTorch | +| `--bf16-unet --bf16-vae --bf16-text-enc` | Keeps models in BF16 — Blackwell's native precision | +| `--dont-upcast-attention` | Keeps attention ops in FP16/BF16 for speed | +| `--disable-mmap` | Combined with the `copy=False` patch, fixes double memory usage on unified memory | + +> **Note:** Do **not** use `--gpu-only`. It forces a split memory model that fights the unified memory fabric on Grace-Blackwell systems. +> **Note:** Do **not** use `--high-vram`. It pins every model in memory causing OOM issues. + +### Environment variables + +| Variable | Purpose | +|---|---| +| `PYTORCH_NO_CUDA_MEMORY_CACHING=1` | Disables PyTorch CUDA caching allocator — lets the unified memory manager handle allocations | +| `CUDA_MANAGED_FORCE_DEVICE_ALLOC=1` | Forces managed memory to prefer device allocation | +| `TORCH_COMPILE_DISABLE=1` / `TORCHDYNAMO_DISABLE=1` | Disables `torch.compile` — Triton does not support `sm_121a` yet | +| `CUDA_MODULE_LOADING=CUDA` | Loads CUDA modules eagerly for faster model loading | +| `OMP_NUM_THREADS=20` | Tuned for the Grace CPU core count | + +--- + +## 📦 Volumes + +| Host path | Container path | Purpose | +|---|---|---| +| `${COMFYUI_HOST_PATH}/models` | `/opt/ComfyUI/models` | Model files (shared with host) | +| `${COMFYUI_DATA_PATH}/custom_nodes` | `/opt/ComfyUI/custom_nodes` | Custom nodes (incl. ComfyUI-Manager) | +| `${COMFYUI_DATA_PATH}/user` | `/opt/ComfyUI/user` | User settings & ComfyUI-Manager config | +| `${COMFYUI_DATA_PATH}/output` | `/opt/ComfyUI/output` | Generated images/videos | +| `${COMFYUI_DATA_PATH}/input` | `/opt/ComfyUI/input` | Input files | +| `${COMFYUI_DATA_PATH}/workflows` | `/opt/ComfyUI/workflows` | Saved workflows | + +--- + +## 📱 ComfyUIMini (Mobile UI) + +[ComfyUIMini](https://github.com/ImDarkTom/ComfyUIMini) is included as an optional second service. It provides a lightweight, mobile-friendly interface that proxies requests to the ComfyUI backend over the internal Docker network. + +It starts automatically after ComfyUI passes its health check and shares the `output` directory for gallery access. + +--- + +## 🩹 Patches Applied + +### `comfy/utils.py` — `copy=False` fix + +The Dockerfile patches one line in ComfyUI's source at build time: + +```python +# Before +tensor = tensor.to(device=device, copy=True) + +# After +tensor = tensor.to(device=device, copy=False) +``` + +On unified memory systems, `copy=True` causes tensors to be duplicated unnecessarily, effectively doubling VRAM usage when `--disable-mmap` is active. Setting `copy=False` eliminates this. + +--- + +## 🔄 Updating ComfyUI + +The `COMFYUI_REF` variable (in `.env` or at build time) controls which commit/tag/branch of ComfyUI is checked out. Pin it for reproducibility: + +```env +COMFYUI_REF=v0.3.43 +``` + +ComfyUI-Manager is **not** baked into the image — it is cloned fresh at container startup into the mounted `custom_nodes` volume, so you always get the latest version. + +--- + +## 📄 License + +MIT