mARTin-B78/faster-whisper-dgx-spark
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Rename and reorganize project files

Browse Source
This commit is contained in:
mARTin 2026-03-26 12:36:51 +01:00
parent 49deee91af
commit 8777e931d8
3 changed files with 74 additions and 173 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

0
docker/Dockerfile → Dockerfile
View File

130
README.md
View File

@ -1,61 +1,67 @@
# whisper-test (faster-whisper on DGX Spark/ThinkStation PGX) # faster-whisper on DGX Spark / ThinkStation PGX
NVIDIA DGX Spark 相当環境（Lenovo ThinkStation PGX）で、`faster-whisper` を CUDA で確実に動かすための最小構成をまとめたリポジトリです。ホスト側 CUDA 13 が既に入っており変更しづらい研究用マシンでも、Docker 上で CUDA 12.4 + cuDNN 9 でビルドした CTranslate2 を用いて GPU 推論を再現できます。 This repository provides a minimal, reproducible setup to run `faster-whisper` with CUDA on an NVIDIA DGX Spark–equivalent environment (Lenovo ThinkStation PGX).
Even if the host already has CUDA 13 installed (and is hard to change), you can run GPU inference reliably by building CTranslate2 with CUDA 12.4 + cuDNN 9 inside Docker.
公式 faster-whisper リポジトリ: https://github.com/SYSTRAN/faster-whisper Official faster-whisper repository: https://github.com/SYSTRAN/faster-whisper
## なぜこの構成が必要か ## Why this setup is needed
- `faster-whisper` は CTranslate2 を利用し、ビルド時の CUDA/cuDNN と実行環境が一致していないと GPU 版が動作しない。 - `faster-whisper` relies on CTranslate2, and the CUDA/cuDNN versions used at build time must match the runtime environment for the GPU build to work.
- ThinkStation PGX ではホストに CUDA 13 が入っており、`pip install ctranslate2` すると CPU 版や CUDA 非対応ビルドが入ることがある。 - On ThinkStation PGX, CUDA 13 is installed on the host. A simple `pip install ctranslate2` may install a CPU-only or non-CUDA build.
- 公式手順だけでは CUDA 版が入らず、`ValueError: This CTranslate2 package was not compiled with CUDA support` となるケースがあった。 - Following only the official instructions sometimes results in `ValueError: This CTranslate2 package was not compiled with CUDA support`.
- そこで CUDA 12.4 + cuDNN 9 を明示した Docker で CTranslate2 をソースビルドし、Python バインディングを明示的に再インストールすることで再現性を確保。 - To ensure reproducibility, we explicitly build CTranslate2 from source with CUDA 12.4 + cuDNN 9 in Docker, and reinstall the Python bindings from that build.
## 提供物 ## What this repo provides
- `docker-compose.yml` と `docker/Dockerfile` による GPU 対応コンテナ構成 - `docker-compose.yml` and `docker/Dockerfile` for a GPU-enabled container:
- ベース: `nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04` - Base image: `nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04`
- CTranslate2 v4.6.2 を CUDA 有効でソースビルド - CTranslate2 v4.6.2 built from source with CUDA enabled
- `pip` で入る CPU 版 `ctranslate2` を一度アンインストールし、ビルド済み Python バインディングを再インストール - The CPU/non-CUDA `ctranslate2` wheel installed via pip is uninstalled once, and then replaced with the locally built CUDA-enabled Python bindings
- FastAPI + Uvicorn の簡易サーバ (`docker/server.py`) - A minimal FastAPI + Uvicorn server (`docker/server.py`):
- `POST /transcribe` で音声を受け取り、Whisper 推論結果を返却 - `POST /transcribe`: receive audio and return Whisper transcription
- `GET /health` でモデル名・デバイス・compute_type を返却 - `GET /health`: return model name, device, and compute_type
- サンプル音声とテストスクリプト（`test/`） - Sample audio and test scripts (`test/`)
## 仕組みのポイント ## Key design points
- CUDA 12.x + cuDNN 9 で CTranslate2 をビルドし、実行時も同一バージョンを保証 - CTranslate2 is built with CUDA 12.x + cuDNN 9, and the same version is guaranteed at runtime via the Docker image.
- `entrypoint.sh` で `LD_LIBRARY_PATH` を明示設定し、`nvidia.cublas.lib` / `nvidia.cudnn.lib` のパスを動的に追加 - `entrypoint.sh` explicitly configures `LD_LIBRARY_PATH` and dynamically adds the paths to `nvidia.cublas.lib` and `nvidia.cudnn.lib`.
- ホスト CUDA 13 と切り離すため Docker を必須とし、NVIDIA Container Toolkit を前提に GPU を渡す - Docker is required to isolate from host CUDA 13; GPU access assumes NVIDIA Container Toolkit.
## 前提条件 ## Prerequisites
- NVIDIA Container Toolkit 導入済み - NVIDIA Container Toolkit installed
- GPU ドライバがホストに正しく入っていること - GPU driver correctly installed on the host
- ホスト側 CUDA は 13 系のままで可（変更不要） - Host CUDA can remain 13.x (no need to change)
- Internet connection (for first-time model download)
### Optional: Local CPU-only run (.venv)
If you want to try CPU-only inference without Docker, you can use a local Python virtual environment:
### ローカルCPU実行（任意）
Docker を使わず手元の Python (.venv) で CPU 推論だけ試す場合の簡易手順:
```bash ```bash
python3 -m venv .venv python3 -m venv .venv
source .venv/bin/activate source .venv/bin/activate
pip install --upgrade pip pip install --upgrade pip
pip install -r docker/requirements.txt pip install -r docker/requirements.txt
# CPU でサンプル音声を実行 # Run sample audio on CPU
FW_DEVICE=cpu FW_COMPUTE_TYPE=int8 python3 test/transcribe_cpu.py FW_DEVICE=cpu FW_COMPUTE_TYPE=int8 python3 test/transcribe_cpu.py
``` ```
※ CPU のみの確認用。GPU を使う場合は Docker 構成を推奨。
## セットアップ This path is intended just for quick CPU checks; for GPU we recommend using the Docker setup below.
## Setup (Docker)
```bash ```bash
# ビルド # Build
docker compose build docker compose build
# 起動（GPU 必須） # Start (GPU required)
docker compose up -d docker compose up -d
# ヘルスチェック # Health check
curl http://localhost:8002/health curl http://localhost:8002/health
``` ```
## 推論APIの使い方 ## How to use the inference API
```bash ```bash
curl -X POST http://localhost:8002/transcribe \ curl -X POST http://localhost:8002/transcribe \
-F "file=@test/003.wav" \ -F "file=@test/003.wav" \
@ -63,37 +69,49 @@ curl -X POST http://localhost:8002/transcribe \
-F "beam_size=5" \ -F "beam_size=5" \
-F "vad_filter=true" -F "vad_filter=true"
``` ```
レスポンスには `text` と `segments`（start/end/text）、`language`, `language_probability` が含まれます。
環境変数でモデルや実行設定を変更できます: The response includes `text`, `segments` (start/end/text), `language`, and `language_probability`.
- `FW_MODEL` (デフォルト `large-v3`)
- `FW_DEVICE` (デフォルト `cuda`) You can change model and runtime settings via environment variables:
- `FW_COMPUTE_TYPE` (デフォルト `float16`) - `FW_MODEL` (default: `large-v3`)
- `FW_DEVICE` (default: `cuda`)
- `FW_COMPUTE_TYPE` (default: `float16`)
### Running `test/` scripts inside the Docker container
If you prefer not to install Python on the host, you can run the test scripts inside the running container:
### Docker コンテナ内で `test/` のスクリプトを実行する例
ホストに Python を入れずに検証したい場合、起動済みコンテナ内でテストスクリプトを実行できます。
```bash ```bash
# 1) コンテナに入る（compose が起動している前提） # 1) Enter the container (assuming docker compose is up)
docker compose exec whisper bash docker compose exec whisper bash
# 2) GPU 版サンプル（デフォルトの large-v3 / float16） # 2) GPU sample (default: large-v3 / float16)
python3 test/transcribe.py python3 test/transcribe.py
# 3) CPU で試したい場合（compute_type を切り替え） # 3) CPU run (switch compute_type via env vars)
FW_DEVICE=cpu FW_COMPUTE_TYPE=int8 python3 test/transcribe_cpu.py FW_DEVICE=cpu FW_COMPUTE_TYPE=int8 python3 test/transcribe_cpu.py
``` ```
いずれも `test/003.wav` を入力にし、標準出力へテキストと各セグメントを表示します。自分の音声で試す場合は `python3 test/transcribe.py /path/to/your.wav` のようにパスを指定してください。
## ディレクトリ構成（抜粋） Both scripts use `test/003.wav` as the default input and print the transcript and segments to stdout.
- `docker-compose.yml` — サービス定義（GPU リクエスト含む） To test with your own audio file, pass the path as an argument:
- `docker/Dockerfile` — CUDA 12.4 + cuDNN 9 ベースで CTranslate2 をソースビルド
- `docker/entrypoint.sh` — CUDA ライブラリパスを動的設定し uvicorn 起動 ```bash
- `docker/server.py` — FastAPI サーバ（/health, /transcribe） python3 test/transcribe.py /path/to/your.wav
- `docker/requirements.txt` — 最小依存: `faster-whisper`, `fastapi`, `uvicorn[standard]`, `python-multipart` ```
- `test/` — サンプル音声とローカル実行スクリプト
## Directory layout (partial)
- `docker-compose.yml` — Service definition (including GPU resources)
- `docker/Dockerfile` — Build CTranslate2 from source on CUDA 12.4 + cuDNN 9
- `docker/entrypoint.sh` — Configure CUDA library paths and start uvicorn
- `docker/server.py` — FastAPI server (`/health`, `/transcribe`)
- `docker/requirements.txt` — Minimal dependencies: `faster-whisper`, `fastapi`, `uvicorn[standard]`, `python-multipart`
- `test/` — Sample audio (`003.wav`) and local test scripts
## Notes for reproducibility
- A simple `pip install faster-whisper` does not guarantee a CUDA-enabled `ctranslate2` build.
- Even if installation succeeds, a CUDA version mismatch may cause the GPU not to be used at runtime.
- This Dockerfile explicitly uninstalls the CPU build once and reinstalls the locally built CUDA-enabled version to keep everything consistent.
## 再現のための注意点
- `pip install faster-whisper` だけでは CUDA 版 `ctranslate2` は保証されない
- CUDA バージョン不一致だとインストール成功しても実行時に GPU を掴まない
- 本 Dockerfile は一度 CPU 版をアンインストールし、ビルド済み CUDA 版を再インストールして整合性を取る

117
README_EN.md
View File

@ -1,117 +0,0 @@
# faster-whisper on DGX Spark / ThinkStation PGX
This repository provides a minimal, reproducible setup to run `faster-whisper` with CUDA on an NVIDIA DGX Spark–equivalent environment (Lenovo ThinkStation PGX).
Even if the host already has CUDA 13 installed (and is hard to change), you can run GPU inference reliably by building CTranslate2 with CUDA 12.4 + cuDNN 9 inside Docker.
Official faster-whisper repository: https://github.com/SYSTRAN/faster-whisper
## Why this setup is needed
- `faster-whisper` relies on CTranslate2, and the CUDA/cuDNN versions used at build time must match the runtime environment for the GPU build to work.
- On ThinkStation PGX, CUDA 13 is installed on the host. A simple `pip install ctranslate2` may install a CPU-only or non-CUDA build.
- Following only the official instructions sometimes results in `ValueError: This CTranslate2 package was not compiled with CUDA support`.
- To ensure reproducibility, we explicitly build CTranslate2 from source with CUDA 12.4 + cuDNN 9 in Docker, and reinstall the Python bindings from that build.
## What this repo provides
- `docker-compose.yml` and `docker/Dockerfile` for a GPU-enabled container:
- Base image: `nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04`
- CTranslate2 v4.6.2 built from source with CUDA enabled
- The CPU/non-CUDA `ctranslate2` wheel installed via pip is uninstalled once, and then replaced with the locally built CUDA-enabled Python bindings
- A minimal FastAPI + Uvicorn server (`docker/server.py`):
- `POST /transcribe`: receive audio and return Whisper transcription
- `GET /health`: return model name, device, and compute_type
- Sample audio and test scripts (`test/`)
## Key design points
- CTranslate2 is built with CUDA 12.x + cuDNN 9, and the same version is guaranteed at runtime via the Docker image.
- `entrypoint.sh` explicitly configures `LD_LIBRARY_PATH` and dynamically adds the paths to `nvidia.cublas.lib` and `nvidia.cudnn.lib`.
- Docker is required to isolate from host CUDA 13; GPU access assumes NVIDIA Container Toolkit.
## Prerequisites
- NVIDIA Container Toolkit installed
- GPU driver correctly installed on the host
- Host CUDA can remain 13.x (no need to change)
- Internet connection (for first-time model download)
### Optional: Local CPU-only run (.venv)
If you want to try CPU-only inference without Docker, you can use a local Python virtual environment:
```bash
python3 -m venv .venv
source .venv/bin/activate
pip install --upgrade pip
pip install -r docker/requirements.txt
# Run sample audio on CPU
FW_DEVICE=cpu FW_COMPUTE_TYPE=int8 python3 test/transcribe_cpu.py
```
This path is intended just for quick CPU checks; for GPU we recommend using the Docker setup below.
## Setup (Docker)
```bash
# Build
docker compose build
# Start (GPU required)
docker compose up -d
# Health check
curl http://localhost:8002/health
```
## How to use the inference API
```bash
curl -X POST http://localhost:8002/transcribe \
-F "file=@test/003.wav" \
-F "language=ja" \
-F "beam_size=5" \
-F "vad_filter=true"
```
The response includes `text`, `segments` (start/end/text), `language`, and `language_probability`.
You can change model and runtime settings via environment variables:
- `FW_MODEL` (default: `large-v3`)
- `FW_DEVICE` (default: `cuda`)
- `FW_COMPUTE_TYPE` (default: `float16`)
### Running `test/` scripts inside the Docker container
If you prefer not to install Python on the host, you can run the test scripts inside the running container:
```bash
# 1) Enter the container (assuming docker compose is up)
docker compose exec whisper bash
# 2) GPU sample (default: large-v3 / float16)
python3 test/transcribe.py
# 3) CPU run (switch compute_type via env vars)
FW_DEVICE=cpu FW_COMPUTE_TYPE=int8 python3 test/transcribe_cpu.py
```
Both scripts use `test/003.wav` as the default input and print the transcript and segments to stdout.
To test with your own audio file, pass the path as an argument:
```bash
python3 test/transcribe.py /path/to/your.wav
```
## Directory layout (partial)
- `docker-compose.yml` — Service definition (including GPU resources)
- `docker/Dockerfile` — Build CTranslate2 from source on CUDA 12.4 + cuDNN 9
- `docker/entrypoint.sh` — Configure CUDA library paths and start uvicorn
- `docker/server.py` — FastAPI server (`/health`, `/transcribe`)
- `docker/requirements.txt` — Minimal dependencies: `faster-whisper`, `fastapi`, `uvicorn[standard]`, `python-multipart`
- `test/` — Sample audio (`003.wav`) and local test scripts
## Notes for reproducibility
- A simple `pip install faster-whisper` does not guarantee a CUDA-enabled `ctranslate2` build.
- Even if installation succeeds, a CUDA version mismatch may cause the GPU not to be used at runtime.
- This Dockerfile explicitly uninstalls the CPU build once and reinstalls the locally built CUDA-enabled version to keep everything consistent.