diff --git a/docker/Dockerfile b/Dockerfile similarity index 100% rename from docker/Dockerfile rename to Dockerfile diff --git a/README.md b/README.md index ecc2fc3..5ab5046 100644 --- a/README.md +++ b/README.md @@ -1,61 +1,67 @@ -# whisper-test (faster-whisper on DGX Spark/ThinkStation PGX) +# faster-whisper on DGX Spark / ThinkStation PGX -NVIDIA DGX Spark 相当環境(Lenovo ThinkStation PGX)で、`faster-whisper` を CUDA で確実に動かすための最小構成をまとめたリポジトリです。ホスト側 CUDA 13 が既に入っており変更しづらい研究用マシンでも、Docker 上で CUDA 12.4 + cuDNN 9 でビルドした CTranslate2 を用いて GPU 推論を再現できます。 +This repository provides a minimal, reproducible setup to run `faster-whisper` with CUDA on an NVIDIA DGX Spark–equivalent environment (Lenovo ThinkStation PGX). +Even if the host already has CUDA 13 installed (and is hard to change), you can run GPU inference reliably by building CTranslate2 with CUDA 12.4 + cuDNN 9 inside Docker. -公式 faster-whisper リポジトリ: https://github.com/SYSTRAN/faster-whisper +Official faster-whisper repository: https://github.com/SYSTRAN/faster-whisper -## なぜこの構成が必要か -- `faster-whisper` は CTranslate2 を利用し、ビルド時の CUDA/cuDNN と実行環境が一致していないと GPU 版が動作しない。 -- ThinkStation PGX ではホストに CUDA 13 が入っており、`pip install ctranslate2` すると CPU 版や CUDA 非対応ビルドが入ることがある。 -- 公式手順だけでは CUDA 版が入らず、`ValueError: This CTranslate2 package was not compiled with CUDA support` となるケースがあった。 -- そこで CUDA 12.4 + cuDNN 9 を明示した Docker で CTranslate2 をソースビルドし、Python バインディングを明示的に再インストールすることで再現性を確保。 +## Why this setup is needed +- `faster-whisper` relies on CTranslate2, and the CUDA/cuDNN versions used at build time must match the runtime environment for the GPU build to work. +- On ThinkStation PGX, CUDA 13 is installed on the host. A simple `pip install ctranslate2` may install a CPU-only or non-CUDA build. +- Following only the official instructions sometimes results in `ValueError: This CTranslate2 package was not compiled with CUDA support`. +- To ensure reproducibility, we explicitly build CTranslate2 from source with CUDA 12.4 + cuDNN 9 in Docker, and reinstall the Python bindings from that build. -## 提供物 -- `docker-compose.yml` と `docker/Dockerfile` による GPU 対応コンテナ構成 - - ベース: `nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04` - - CTranslate2 v4.6.2 を CUDA 有効でソースビルド - - `pip` で入る CPU 版 `ctranslate2` を一度アンインストールし、ビルド済み Python バインディングを再インストール -- FastAPI + Uvicorn の簡易サーバ (`docker/server.py`) - - `POST /transcribe` で音声を受け取り、Whisper 推論結果を返却 - - `GET /health` でモデル名・デバイス・compute_type を返却 -- サンプル音声とテストスクリプト(`test/`) +## What this repo provides +- `docker-compose.yml` and `docker/Dockerfile` for a GPU-enabled container: + - Base image: `nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04` + - CTranslate2 v4.6.2 built from source with CUDA enabled + - The CPU/non-CUDA `ctranslate2` wheel installed via pip is uninstalled once, and then replaced with the locally built CUDA-enabled Python bindings +- A minimal FastAPI + Uvicorn server (`docker/server.py`): + - `POST /transcribe`: receive audio and return Whisper transcription + - `GET /health`: return model name, device, and compute_type +- Sample audio and test scripts (`test/`) -## 仕組みのポイント -- CUDA 12.x + cuDNN 9 で CTranslate2 をビルドし、実行時も同一バージョンを保証 -- `entrypoint.sh` で `LD_LIBRARY_PATH` を明示設定し、`nvidia.cublas.lib` / `nvidia.cudnn.lib` のパスを動的に追加 -- ホスト CUDA 13 と切り離すため Docker を必須とし、NVIDIA Container Toolkit を前提に GPU を渡す +## Key design points +- CTranslate2 is built with CUDA 12.x + cuDNN 9, and the same version is guaranteed at runtime via the Docker image. +- `entrypoint.sh` explicitly configures `LD_LIBRARY_PATH` and dynamically adds the paths to `nvidia.cublas.lib` and `nvidia.cudnn.lib`. +- Docker is required to isolate from host CUDA 13; GPU access assumes NVIDIA Container Toolkit. -## 前提条件 -- NVIDIA Container Toolkit 導入済み -- GPU ドライバがホストに正しく入っていること -- ホスト側 CUDA は 13 系のままで可(変更不要) +## Prerequisites +- NVIDIA Container Toolkit installed +- GPU driver correctly installed on the host +- Host CUDA can remain 13.x (no need to change) +- Internet connection (for first-time model download) + +### Optional: Local CPU-only run (.venv) +If you want to try CPU-only inference without Docker, you can use a local Python virtual environment: -### ローカルCPU実行(任意) -Docker を使わず手元の Python (.venv) で CPU 推論だけ試す場合の簡易手順: ```bash python3 -m venv .venv source .venv/bin/activate pip install --upgrade pip pip install -r docker/requirements.txt -# CPU でサンプル音声を実行 +# Run sample audio on CPU FW_DEVICE=cpu FW_COMPUTE_TYPE=int8 python3 test/transcribe_cpu.py ``` -※ CPU のみの確認用。GPU を使う場合は Docker 構成を推奨。 -## セットアップ +This path is intended just for quick CPU checks; for GPU we recommend using the Docker setup below. + +## Setup (Docker) + ```bash -# ビルド +# Build docker compose build -# 起動(GPU 必須) +# Start (GPU required) docker compose up -d -# ヘルスチェック +# Health check curl http://localhost:8002/health ``` -## 推論APIの使い方 +## How to use the inference API + ```bash curl -X POST http://localhost:8002/transcribe \ -F "file=@test/003.wav" \ @@ -63,37 +69,49 @@ curl -X POST http://localhost:8002/transcribe \ -F "beam_size=5" \ -F "vad_filter=true" ``` -レスポンスには `text` と `segments`(start/end/text)、`language`, `language_probability` が含まれます。 -環境変数でモデルや実行設定を変更できます: -- `FW_MODEL` (デフォルト `large-v3`) -- `FW_DEVICE` (デフォルト `cuda`) -- `FW_COMPUTE_TYPE` (デフォルト `float16`) +The response includes `text`, `segments` (start/end/text), `language`, and `language_probability`. + +You can change model and runtime settings via environment variables: +- `FW_MODEL` (default: `large-v3`) +- `FW_DEVICE` (default: `cuda`) +- `FW_COMPUTE_TYPE` (default: `float16`) + +### Running `test/` scripts inside the Docker container + +If you prefer not to install Python on the host, you can run the test scripts inside the running container: -### Docker コンテナ内で `test/` のスクリプトを実行する例 -ホストに Python を入れずに検証したい場合、起動済みコンテナ内でテストスクリプトを実行できます。 ```bash -# 1) コンテナに入る(compose が起動している前提) +# 1) Enter the container (assuming docker compose is up) docker compose exec whisper bash -# 2) GPU 版サンプル(デフォルトの large-v3 / float16) +# 2) GPU sample (default: large-v3 / float16) python3 test/transcribe.py -# 3) CPU で試したい場合(compute_type を切り替え) +# 3) CPU run (switch compute_type via env vars) FW_DEVICE=cpu FW_COMPUTE_TYPE=int8 python3 test/transcribe_cpu.py ``` -いずれも `test/003.wav` を入力にし、標準出力へテキストと各セグメントを表示します。自分の音声で試す場合は `python3 test/transcribe.py /path/to/your.wav` のようにパスを指定してください。 -## ディレクトリ構成(抜粋) -- `docker-compose.yml` — サービス定義(GPU リクエスト含む) -- `docker/Dockerfile` — CUDA 12.4 + cuDNN 9 ベースで CTranslate2 をソースビルド -- `docker/entrypoint.sh` — CUDA ライブラリパスを動的設定し uvicorn 起動 -- `docker/server.py` — FastAPI サーバ(/health, /transcribe) -- `docker/requirements.txt` — 最小依存: `faster-whisper`, `fastapi`, `uvicorn[standard]`, `python-multipart` -- `test/` — サンプル音声とローカル実行スクリプト +Both scripts use `test/003.wav` as the default input and print the transcript and segments to stdout. +To test with your own audio file, pass the path as an argument: + +```bash +python3 test/transcribe.py /path/to/your.wav +``` + +## Directory layout (partial) + +- `docker-compose.yml` — Service definition (including GPU resources) +- `docker/Dockerfile` — Build CTranslate2 from source on CUDA 12.4 + cuDNN 9 +- `docker/entrypoint.sh` — Configure CUDA library paths and start uvicorn +- `docker/server.py` — FastAPI server (`/health`, `/transcribe`) +- `docker/requirements.txt` — Minimal dependencies: `faster-whisper`, `fastapi`, `uvicorn[standard]`, `python-multipart` +- `test/` — Sample audio (`003.wav`) and local test scripts + +## Notes for reproducibility + +- A simple `pip install faster-whisper` does not guarantee a CUDA-enabled `ctranslate2` build. +- Even if installation succeeds, a CUDA version mismatch may cause the GPU not to be used at runtime. +- This Dockerfile explicitly uninstalls the CPU build once and reinstalls the locally built CUDA-enabled version to keep everything consistent. -## 再現のための注意点 -- `pip install faster-whisper` だけでは CUDA 版 `ctranslate2` は保証されない -- CUDA バージョン不一致だとインストール成功しても実行時に GPU を掴まない -- 本 Dockerfile は一度 CPU 版をアンインストールし、ビルド済み CUDA 版を再インストールして整合性を取る diff --git a/README_EN.md b/README_EN.md deleted file mode 100644 index 5ab5046..0000000 --- a/README_EN.md +++ /dev/null @@ -1,117 +0,0 @@ -# faster-whisper on DGX Spark / ThinkStation PGX - -This repository provides a minimal, reproducible setup to run `faster-whisper` with CUDA on an NVIDIA DGX Spark–equivalent environment (Lenovo ThinkStation PGX). -Even if the host already has CUDA 13 installed (and is hard to change), you can run GPU inference reliably by building CTranslate2 with CUDA 12.4 + cuDNN 9 inside Docker. - -Official faster-whisper repository: https://github.com/SYSTRAN/faster-whisper - -## Why this setup is needed -- `faster-whisper` relies on CTranslate2, and the CUDA/cuDNN versions used at build time must match the runtime environment for the GPU build to work. -- On ThinkStation PGX, CUDA 13 is installed on the host. A simple `pip install ctranslate2` may install a CPU-only or non-CUDA build. -- Following only the official instructions sometimes results in `ValueError: This CTranslate2 package was not compiled with CUDA support`. -- To ensure reproducibility, we explicitly build CTranslate2 from source with CUDA 12.4 + cuDNN 9 in Docker, and reinstall the Python bindings from that build. - -## What this repo provides -- `docker-compose.yml` and `docker/Dockerfile` for a GPU-enabled container: - - Base image: `nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04` - - CTranslate2 v4.6.2 built from source with CUDA enabled - - The CPU/non-CUDA `ctranslate2` wheel installed via pip is uninstalled once, and then replaced with the locally built CUDA-enabled Python bindings -- A minimal FastAPI + Uvicorn server (`docker/server.py`): - - `POST /transcribe`: receive audio and return Whisper transcription - - `GET /health`: return model name, device, and compute_type -- Sample audio and test scripts (`test/`) - -## Key design points -- CTranslate2 is built with CUDA 12.x + cuDNN 9, and the same version is guaranteed at runtime via the Docker image. -- `entrypoint.sh` explicitly configures `LD_LIBRARY_PATH` and dynamically adds the paths to `nvidia.cublas.lib` and `nvidia.cudnn.lib`. -- Docker is required to isolate from host CUDA 13; GPU access assumes NVIDIA Container Toolkit. - -## Prerequisites -- NVIDIA Container Toolkit installed -- GPU driver correctly installed on the host -- Host CUDA can remain 13.x (no need to change) -- Internet connection (for first-time model download) - -### Optional: Local CPU-only run (.venv) -If you want to try CPU-only inference without Docker, you can use a local Python virtual environment: - -```bash -python3 -m venv .venv -source .venv/bin/activate -pip install --upgrade pip -pip install -r docker/requirements.txt - -# Run sample audio on CPU -FW_DEVICE=cpu FW_COMPUTE_TYPE=int8 python3 test/transcribe_cpu.py -``` - -This path is intended just for quick CPU checks; for GPU we recommend using the Docker setup below. - -## Setup (Docker) - -```bash -# Build -docker compose build - -# Start (GPU required) -docker compose up -d - -# Health check -curl http://localhost:8002/health -``` - -## How to use the inference API - -```bash -curl -X POST http://localhost:8002/transcribe \ - -F "file=@test/003.wav" \ - -F "language=ja" \ - -F "beam_size=5" \ - -F "vad_filter=true" -``` - -The response includes `text`, `segments` (start/end/text), `language`, and `language_probability`. - -You can change model and runtime settings via environment variables: -- `FW_MODEL` (default: `large-v3`) -- `FW_DEVICE` (default: `cuda`) -- `FW_COMPUTE_TYPE` (default: `float16`) - -### Running `test/` scripts inside the Docker container - -If you prefer not to install Python on the host, you can run the test scripts inside the running container: - -```bash -# 1) Enter the container (assuming docker compose is up) -docker compose exec whisper bash - -# 2) GPU sample (default: large-v3 / float16) -python3 test/transcribe.py - -# 3) CPU run (switch compute_type via env vars) -FW_DEVICE=cpu FW_COMPUTE_TYPE=int8 python3 test/transcribe_cpu.py -``` - -Both scripts use `test/003.wav` as the default input and print the transcript and segments to stdout. -To test with your own audio file, pass the path as an argument: - -```bash -python3 test/transcribe.py /path/to/your.wav -``` - -## Directory layout (partial) - -- `docker-compose.yml` — Service definition (including GPU resources) -- `docker/Dockerfile` — Build CTranslate2 from source on CUDA 12.4 + cuDNN 9 -- `docker/entrypoint.sh` — Configure CUDA library paths and start uvicorn -- `docker/server.py` — FastAPI server (`/health`, `/transcribe`) -- `docker/requirements.txt` — Minimal dependencies: `faster-whisper`, `fastapi`, `uvicorn[standard]`, `python-multipart` -- `test/` — Sample audio (`003.wav`) and local test scripts - -## Notes for reproducibility - -- A simple `pip install faster-whisper` does not guarantee a CUDA-enabled `ctranslate2` build. -- Even if installation succeeds, a CUDA version mismatch may cause the GPU not to be used at runtime. -- This Dockerfile explicitly uninstalls the CPU build once and reinstalls the locally built CUDA-enabled version to keep everything consistent. - -