Rename and reorganize project files
This commit is contained in:
parent
49deee91af
commit
8777e931d8
130
README.md
130
README.md
@ -1,61 +1,67 @@
|
||||
# whisper-test (faster-whisper on DGX Spark/ThinkStation PGX)
|
||||
# faster-whisper on DGX Spark / ThinkStation PGX
|
||||
|
||||
NVIDIA DGX Spark 相当環境（Lenovo ThinkStation PGX）で、`faster-whisper` を CUDA で確実に動かすための最小構成をまとめたリポジトリです。ホスト側 CUDA 13 が既に入っており変更しづらい研究用マシンでも、Docker 上で CUDA 12.4 + cuDNN 9 でビルドした CTranslate2 を用いて GPU 推論を再現できます。
|
||||
This repository provides a minimal, reproducible setup to run `faster-whisper` with CUDA on an NVIDIA DGX Spark–equivalent environment (Lenovo ThinkStation PGX).
|
||||
Even if the host already has CUDA 13 installed (and is hard to change), you can run GPU inference reliably by building CTranslate2 with CUDA 12.4 + cuDNN 9 inside Docker.
|
||||
|
||||
公式 faster-whisper リポジトリ: https://github.com/SYSTRAN/faster-whisper
|
||||
Official faster-whisper repository: https://github.com/SYSTRAN/faster-whisper
|
||||
|
||||
## なぜこの構成が必要か
|
||||
- `faster-whisper` は CTranslate2 を利用し、ビルド時の CUDA/cuDNN と実行環境が一致していないと GPU 版が動作しない。
|
||||
- ThinkStation PGX ではホストに CUDA 13 が入っており、`pip install ctranslate2` すると CPU 版や CUDA 非対応ビルドが入ることがある。
|
||||
- 公式手順だけでは CUDA 版が入らず、`ValueError: This CTranslate2 package was not compiled with CUDA support` となるケースがあった。
|
||||
- そこで CUDA 12.4 + cuDNN 9 を明示した Docker で CTranslate2 をソースビルドし、Python バインディングを明示的に再インストールすることで再現性を確保。
|
||||
## Why this setup is needed
|
||||
- `faster-whisper` relies on CTranslate2, and the CUDA/cuDNN versions used at build time must match the runtime environment for the GPU build to work.
|
||||
- On ThinkStation PGX, CUDA 13 is installed on the host. A simple `pip install ctranslate2` may install a CPU-only or non-CUDA build.
|
||||
- Following only the official instructions sometimes results in `ValueError: This CTranslate2 package was not compiled with CUDA support`.
|
||||
- To ensure reproducibility, we explicitly build CTranslate2 from source with CUDA 12.4 + cuDNN 9 in Docker, and reinstall the Python bindings from that build.
|
||||
|
||||
## 提供物
|
||||
- `docker-compose.yml` と `docker/Dockerfile` による GPU 対応コンテナ構成
|
||||
- ベース: `nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04`
|
||||
- CTranslate2 v4.6.2 を CUDA 有効でソースビルド
|
||||
- `pip` で入る CPU 版 `ctranslate2` を一度アンインストールし、ビルド済み Python バインディングを再インストール
|
||||
- FastAPI + Uvicorn の簡易サーバ (`docker/server.py`)
|
||||
- `POST /transcribe` で音声を受け取り、Whisper 推論結果を返却
|
||||
- `GET /health` でモデル名・デバイス・compute_type を返却
|
||||
- サンプル音声とテストスクリプト（`test/`）
|
||||
## What this repo provides
|
||||
- `docker-compose.yml` and `docker/Dockerfile` for a GPU-enabled container:
|
||||
- Base image: `nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04`
|
||||
- CTranslate2 v4.6.2 built from source with CUDA enabled
|
||||
- The CPU/non-CUDA `ctranslate2` wheel installed via pip is uninstalled once, and then replaced with the locally built CUDA-enabled Python bindings
|
||||
- A minimal FastAPI + Uvicorn server (`docker/server.py`):
|
||||
- `POST /transcribe`: receive audio and return Whisper transcription
|
||||
- `GET /health`: return model name, device, and compute_type
|
||||
- Sample audio and test scripts (`test/`)
|
||||
|
||||
## 仕組みのポイント
|
||||
- CUDA 12.x + cuDNN 9 で CTranslate2 をビルドし、実行時も同一バージョンを保証
|
||||
- `entrypoint.sh` で `LD_LIBRARY_PATH` を明示設定し、`nvidia.cublas.lib` / `nvidia.cudnn.lib` のパスを動的に追加
|
||||
- ホスト CUDA 13 と切り離すため Docker を必須とし、NVIDIA Container Toolkit を前提に GPU を渡す
|
||||
## Key design points
|
||||
- CTranslate2 is built with CUDA 12.x + cuDNN 9, and the same version is guaranteed at runtime via the Docker image.
|
||||
- `entrypoint.sh` explicitly configures `LD_LIBRARY_PATH` and dynamically adds the paths to `nvidia.cublas.lib` and `nvidia.cudnn.lib`.
|
||||
- Docker is required to isolate from host CUDA 13; GPU access assumes NVIDIA Container Toolkit.
|
||||
|
||||
## 前提条件
|
||||
- NVIDIA Container Toolkit 導入済み
|
||||
- GPU ドライバがホストに正しく入っていること
|
||||
- ホスト側 CUDA は 13 系のままで可（変更不要）
|
||||
## Prerequisites
|
||||
- NVIDIA Container Toolkit installed
|
||||
- GPU driver correctly installed on the host
|
||||
- Host CUDA can remain 13.x (no need to change)
|
||||
- Internet connection (for first-time model download)
|
||||
|
||||
### Optional: Local CPU-only run (.venv)
|
||||
If you want to try CPU-only inference without Docker, you can use a local Python virtual environment:
|
||||
|
||||
### ローカルCPU実行（任意）
|
||||
Docker を使わず手元の Python (.venv) で CPU 推論だけ試す場合の簡易手順:
|
||||
```bash
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install --upgrade pip
|
||||
pip install -r docker/requirements.txt
|
||||
|
||||
# CPU でサンプル音声を実行
|
||||
# Run sample audio on CPU
|
||||
FW_DEVICE=cpu FW_COMPUTE_TYPE=int8 python3 test/transcribe_cpu.py
|
||||
```
|
||||
※ CPU のみの確認用。GPU を使う場合は Docker 構成を推奨。
|
||||
|
||||
## セットアップ
|
||||
This path is intended just for quick CPU checks; for GPU we recommend using the Docker setup below.
|
||||
|
||||
## Setup (Docker)
|
||||
|
||||
```bash
|
||||
# ビルド
|
||||
# Build
|
||||
docker compose build
|
||||
|
||||
# 起動（GPU 必須）
|
||||
# Start (GPU required)
|
||||
docker compose up -d
|
||||
|
||||
# ヘルスチェック
|
||||
# Health check
|
||||
curl http://localhost:8002/health
|
||||
```
|
||||
|
||||
## 推論APIの使い方
|
||||
## How to use the inference API
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8002/transcribe \
|
||||
-F "file=@test/003.wav" \
|
||||
@ -63,37 +69,49 @@ curl -X POST http://localhost:8002/transcribe \
|
||||
-F "beam_size=5" \
|
||||
-F "vad_filter=true"
|
||||
```
|
||||
レスポンスには `text` と `segments`（start/end/text）、`language`, `language_probability` が含まれます。
|
||||
|
||||
環境変数でモデルや実行設定を変更できます:
|
||||
- `FW_MODEL` (デフォルト `large-v3`)
|
||||
- `FW_DEVICE` (デフォルト `cuda`)
|
||||
- `FW_COMPUTE_TYPE` (デフォルト `float16`)
|
||||
The response includes `text`, `segments` (start/end/text), `language`, and `language_probability`.
|
||||
|
||||
You can change model and runtime settings via environment variables:
|
||||
- `FW_MODEL` (default: `large-v3`)
|
||||
- `FW_DEVICE` (default: `cuda`)
|
||||
- `FW_COMPUTE_TYPE` (default: `float16`)
|
||||
|
||||
### Running `test/` scripts inside the Docker container
|
||||
|
||||
If you prefer not to install Python on the host, you can run the test scripts inside the running container:
|
||||
|
||||
### Docker コンテナ内で `test/` のスクリプトを実行する例
|
||||
ホストに Python を入れずに検証したい場合、起動済みコンテナ内でテストスクリプトを実行できます。
|
||||
```bash
|
||||
# 1) コンテナに入る（compose が起動している前提）
|
||||
# 1) Enter the container (assuming docker compose is up)
|
||||
docker compose exec whisper bash
|
||||
|
||||
# 2) GPU 版サンプル（デフォルトの large-v3 / float16）
|
||||
# 2) GPU sample (default: large-v3 / float16)
|
||||
python3 test/transcribe.py
|
||||
|
||||
# 3) CPU で試したい場合（compute_type を切り替え）
|
||||
# 3) CPU run (switch compute_type via env vars)
|
||||
FW_DEVICE=cpu FW_COMPUTE_TYPE=int8 python3 test/transcribe_cpu.py
|
||||
```
|
||||
いずれも `test/003.wav` を入力にし、標準出力へテキストと各セグメントを表示します。自分の音声で試す場合は `python3 test/transcribe.py /path/to/your.wav` のようにパスを指定してください。
|
||||
|
||||
## ディレクトリ構成（抜粋）
|
||||
- `docker-compose.yml` — サービス定義（GPU リクエスト含む）
|
||||
- `docker/Dockerfile` — CUDA 12.4 + cuDNN 9 ベースで CTranslate2 をソースビルド
|
||||
- `docker/entrypoint.sh` — CUDA ライブラリパスを動的設定し uvicorn 起動
|
||||
- `docker/server.py` — FastAPI サーバ（/health, /transcribe）
|
||||
- `docker/requirements.txt` — 最小依存: `faster-whisper`, `fastapi`, `uvicorn[standard]`, `python-multipart`
|
||||
- `test/` — サンプル音声とローカル実行スクリプト
|
||||
Both scripts use `test/003.wav` as the default input and print the transcript and segments to stdout.
|
||||
To test with your own audio file, pass the path as an argument:
|
||||
|
||||
```bash
|
||||
python3 test/transcribe.py /path/to/your.wav
|
||||
```
|
||||
|
||||
## Directory layout (partial)
|
||||
|
||||
- `docker-compose.yml` — Service definition (including GPU resources)
|
||||
- `docker/Dockerfile` — Build CTranslate2 from source on CUDA 12.4 + cuDNN 9
|
||||
- `docker/entrypoint.sh` — Configure CUDA library paths and start uvicorn
|
||||
- `docker/server.py` — FastAPI server (`/health`, `/transcribe`)
|
||||
- `docker/requirements.txt` — Minimal dependencies: `faster-whisper`, `fastapi`, `uvicorn[standard]`, `python-multipart`
|
||||
- `test/` — Sample audio (`003.wav`) and local test scripts
|
||||
|
||||
## Notes for reproducibility
|
||||
|
||||
- A simple `pip install faster-whisper` does not guarantee a CUDA-enabled `ctranslate2` build.
|
||||
- Even if installation succeeds, a CUDA version mismatch may cause the GPU not to be used at runtime.
|
||||
- This Dockerfile explicitly uninstalls the CPU build once and reinstalls the locally built CUDA-enabled version to keep everything consistent.
|
||||
|
||||
## 再現のための注意点
|
||||
- `pip install faster-whisper` だけでは CUDA 版 `ctranslate2` は保証されない
|
||||
- CUDA バージョン不一致だとインストール成功しても実行時に GPU を掴まない
|
||||
- 本 Dockerfile は一度 CPU 版をアンインストールし、ビルド済み CUDA 版を再インストールして整合性を取る
|
||||
|
||||
|
||||
117
README_EN.md
117
README_EN.md
@ -1,117 +0,0 @@
|
||||
# faster-whisper on DGX Spark / ThinkStation PGX
|
||||
|
||||
This repository provides a minimal, reproducible setup to run `faster-whisper` with CUDA on an NVIDIA DGX Spark–equivalent environment (Lenovo ThinkStation PGX).
|
||||
Even if the host already has CUDA 13 installed (and is hard to change), you can run GPU inference reliably by building CTranslate2 with CUDA 12.4 + cuDNN 9 inside Docker.
|
||||
|
||||
Official faster-whisper repository: https://github.com/SYSTRAN/faster-whisper
|
||||
|
||||
## Why this setup is needed
|
||||
- `faster-whisper` relies on CTranslate2, and the CUDA/cuDNN versions used at build time must match the runtime environment for the GPU build to work.
|
||||
- On ThinkStation PGX, CUDA 13 is installed on the host. A simple `pip install ctranslate2` may install a CPU-only or non-CUDA build.
|
||||
- Following only the official instructions sometimes results in `ValueError: This CTranslate2 package was not compiled with CUDA support`.
|
||||
- To ensure reproducibility, we explicitly build CTranslate2 from source with CUDA 12.4 + cuDNN 9 in Docker, and reinstall the Python bindings from that build.
|
||||
|
||||
## What this repo provides
|
||||
- `docker-compose.yml` and `docker/Dockerfile` for a GPU-enabled container:
|
||||
- Base image: `nvidia/cuda:12.4.1-cudnn-devel-ubuntu22.04`
|
||||
- CTranslate2 v4.6.2 built from source with CUDA enabled
|
||||
- The CPU/non-CUDA `ctranslate2` wheel installed via pip is uninstalled once, and then replaced with the locally built CUDA-enabled Python bindings
|
||||
- A minimal FastAPI + Uvicorn server (`docker/server.py`):
|
||||
- `POST /transcribe`: receive audio and return Whisper transcription
|
||||
- `GET /health`: return model name, device, and compute_type
|
||||
- Sample audio and test scripts (`test/`)
|
||||
|
||||
## Key design points
|
||||
- CTranslate2 is built with CUDA 12.x + cuDNN 9, and the same version is guaranteed at runtime via the Docker image.
|
||||
- `entrypoint.sh` explicitly configures `LD_LIBRARY_PATH` and dynamically adds the paths to `nvidia.cublas.lib` and `nvidia.cudnn.lib`.
|
||||
- Docker is required to isolate from host CUDA 13; GPU access assumes NVIDIA Container Toolkit.
|
||||
|
||||
## Prerequisites
|
||||
- NVIDIA Container Toolkit installed
|
||||
- GPU driver correctly installed on the host
|
||||
- Host CUDA can remain 13.x (no need to change)
|
||||
- Internet connection (for first-time model download)
|
||||
|
||||
### Optional: Local CPU-only run (.venv)
|
||||
If you want to try CPU-only inference without Docker, you can use a local Python virtual environment:
|
||||
|
||||
```bash
|
||||
python3 -m venv .venv
|
||||
source .venv/bin/activate
|
||||
pip install --upgrade pip
|
||||
pip install -r docker/requirements.txt
|
||||
|
||||
# Run sample audio on CPU
|
||||
FW_DEVICE=cpu FW_COMPUTE_TYPE=int8 python3 test/transcribe_cpu.py
|
||||
```
|
||||
|
||||
This path is intended just for quick CPU checks; for GPU we recommend using the Docker setup below.
|
||||
|
||||
## Setup (Docker)
|
||||
|
||||
```bash
|
||||
# Build
|
||||
docker compose build
|
||||
|
||||
# Start (GPU required)
|
||||
docker compose up -d
|
||||
|
||||
# Health check
|
||||
curl http://localhost:8002/health
|
||||
```
|
||||
|
||||
## How to use the inference API
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8002/transcribe \
|
||||
-F "file=@test/003.wav" \
|
||||
-F "language=ja" \
|
||||
-F "beam_size=5" \
|
||||
-F "vad_filter=true"
|
||||
```
|
||||
|
||||
The response includes `text`, `segments` (start/end/text), `language`, and `language_probability`.
|
||||
|
||||
You can change model and runtime settings via environment variables:
|
||||
- `FW_MODEL` (default: `large-v3`)
|
||||
- `FW_DEVICE` (default: `cuda`)
|
||||
- `FW_COMPUTE_TYPE` (default: `float16`)
|
||||
|
||||
### Running `test/` scripts inside the Docker container
|
||||
|
||||
If you prefer not to install Python on the host, you can run the test scripts inside the running container:
|
||||
|
||||
```bash
|
||||
# 1) Enter the container (assuming docker compose is up)
|
||||
docker compose exec whisper bash
|
||||
|
||||
# 2) GPU sample (default: large-v3 / float16)
|
||||
python3 test/transcribe.py
|
||||
|
||||
# 3) CPU run (switch compute_type via env vars)
|
||||
FW_DEVICE=cpu FW_COMPUTE_TYPE=int8 python3 test/transcribe_cpu.py
|
||||
```
|
||||
|
||||
Both scripts use `test/003.wav` as the default input and print the transcript and segments to stdout.
|
||||
To test with your own audio file, pass the path as an argument:
|
||||
|
||||
```bash
|
||||
python3 test/transcribe.py /path/to/your.wav
|
||||
```
|
||||
|
||||
## Directory layout (partial)
|
||||
|
||||
- `docker-compose.yml` — Service definition (including GPU resources)
|
||||
- `docker/Dockerfile` — Build CTranslate2 from source on CUDA 12.4 + cuDNN 9
|
||||
- `docker/entrypoint.sh` — Configure CUDA library paths and start uvicorn
|
||||
- `docker/server.py` — FastAPI server (`/health`, `/transcribe`)
|
||||
- `docker/requirements.txt` — Minimal dependencies: `faster-whisper`, `fastapi`, `uvicorn[standard]`, `python-multipart`
|
||||
- `test/` — Sample audio (`003.wav`) and local test scripts
|
||||
|
||||
## Notes for reproducibility
|
||||
|
||||
- A simple `pip install faster-whisper` does not guarantee a CUDA-enabled `ctranslate2` build.
|
||||
- Even if installation succeeds, a CUDA version mismatch may cause the GPU not to be used at runtime.
|
||||
- This Dockerfile explicitly uninstalls the CPU build once and reinstalls the locally built CUDA-enabled version to keep everything consistent.
|
||||
|
||||
|
||||
Loading…
Reference in New Issue
Block a user