diff --git a/CHANGELOG.md b/CHANGELOG.md index 85d96f8..9dab88b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,6 +7,7 @@ Follows [Keep a Changelog](https://keepachangelog.com/en/1.0.0/) ยท versioned wi ## [Unreleased] ### Added +- **QA: Python tests in a throwaway container** (`npm run qa -- --only pyunit --container`): runs the unit tests on production's Python 3.11 in a no-network, read-only container built from the app image (`docker/Dockerfile.test`, adds only `httpx2`); opt-in, SKIPPED when docker or the image is missing. - **Sonar triage `--drain`:** works through open agent issues one at a time, fires the multica webhook, waits for the agent's answer on GitHub and resends once if nothing arrives; stops on a stalled issue. - **Sonar triage can fire a multica webhook autopilot** after each GitHub issue (URL from `~/.config/multica-webhook` or `MULTICA_WEBHOOK_URL`, passed to curl on stdin only); the multica CLI stays as fallback. ### Changed diff --git a/architecture/qa.md b/architecture/qa.md index 67806ac..d9106e1 100644 --- a/architecture/qa.md +++ b/architecture/qa.md @@ -54,3 +54,17 @@ Playwright `baseURL` defaults to the live container (`http://localhost:7890`, ov ## Python interpreter `scripts/qa.mjs` runs Python through the first interpreter that can `import fastapi`: `$PYTHON`, `python3` on PATH, `~/miniforge3/bin/python3`, `/usr/bin/python3`. Agents started by the multica daemon often see `/usr/bin/python3` first (no fastapi), which made their unit tests fail on import; `npm run qa` now works there too. + +## Python tests in a throwaway container +Run: `npm run qa -- --only pyunit --container` (or `QA_PY_CONTAINER=1`). Default stays the host interpreter. + +Why: production runs Python 3.11 with the dependencies installed from `requirements.txt` at image build time (unpinned, so newer than a host's: e.g. starlette 1.7 in the image vs 1.3 on miniforge). The container run tests that exact combination and never touches the live service. + +Setup (once, local only, nothing pushed): `docker build -f docker/Dockerfile.test -t tts-voice-creator-test:local docker`. `docker/Dockerfile.test` is `FROM` the app image plus `httpx2` (needed by `starlette.testclient`, not shipped in production). Without docker or the image the stage reports SKIPPED with the build command. + +Container: `docker run --rm --network none --read-only --tmpfs /tmp:rw,exec --tmpfs /home/app:rw`, `HOME`/`CONFIG_DIR`/`TMPDIR` under `/tmp`, repo mounted read-only at `/work`, user `app`, entrypoint `python3`. + +Known differences from the host run (1086 tests, all OK on host; container: 2 errors, 3 skipped): +- `test_asset_versions...build_id_changes_when_a_script_changes`: the test calls `os.utime` on a repo file; fails on the read-only mount (the mount is deliberate). +- `test_character_identity...browser_and_server_compute_identical_fingerprints`: needs `node`, which the app image does not contain. +Neither is a Python 3.11 vs 3.12 behaviour difference; the other tests pass on 3.11. Coverage (`--sonar`) still uses the host interpreter. diff --git a/docker/Dockerfile.test b/docker/Dockerfile.test new file mode 100644 index 0000000..a5ae842 --- /dev/null +++ b/docker/Dockerfile.test @@ -0,0 +1,8 @@ +# Throwaway test image: the production app image plus test-only dependencies. +# Build: docker build -f docker/Dockerfile.test -t tts-voice-creator-test:local docker +# Local only; never pushed. The app image itself is not modified. +FROM tts-voice-creator-clone-and-design-2:latest +USER root +# starlette.testclient (fastapi TestClient) needs an HTTP client package that production does not ship. +RUN pip install --no-cache-dir httpx2 +USER app diff --git a/scripts/qa.mjs b/scripts/qa.mjs index fbc1f47..97148c7 100644 --- a/scripts/qa.mjs +++ b/scripts/qa.mjs @@ -1,5 +1,5 @@ #!/usr/bin/env node -// Central QA command: `npm run qa [-- --fast] [--sonar] [--only ] [--grep ] [--list]`. +// Central QA command: `npm run qa [-- --fast] [--sonar] [--only ] [--container] [--grep ] [--list]`. // Dependency-free; uses only tools that are already installed and reports the rest as SKIPPED. // Secrets: SONAR_TOKEN / SONAR_HOST_URL come from the environment only and are never printed or put in argv. import { spawnSync } from 'node:child_process'; @@ -13,6 +13,7 @@ const root = join(dirname(fileURLToPath(import.meta.url)), '..'); const args = process.argv.slice(2); const flag = n => args.includes(n); const opt = n => { const i = args.indexOf(n); return i >= 0 ? args[i + 1] : undefined; }; +const container = flag('--container') || process.env.QA_PY_CONTAINER === '1'; const fast = flag('--fast'), withSonar = flag('--sonar'), only = opt('--only'), grep = opt('--grep'); const sh = (cmd, a, o = {}) => spawnSync(cmd, a, { cwd: root, encoding: 'utf8', maxBuffer: 64 << 20, ...o }); @@ -30,6 +31,16 @@ const PY = (() => { const cands = [process.env.PYTHON, 'python3', home && `${home}/miniforge3/bin/python3`, '/usr/bin/python3'].filter(Boolean); return cands.find((c) => sh(c, ['-c', 'import fastapi']).status === 0) || 'python3'; })(); +// Opt-in (--container): run Python tests in a throwaway container built from the app image (production Python), repo mounted read-only. +const TEST_IMAGE = 'tts-voice-creator-test:local'; +const containerArgs = (...cmd) => ['run', '--rm', '--network', 'none', '--read-only', '--tmpfs', '/tmp:rw,exec', '--tmpfs', '/home/app:rw', + '-e', 'HOME=/tmp/home', '-e', 'CONFIG_DIR=/tmp/cfg', '-e', 'TMPDIR=/tmp', '-e', 'PYTHONDONTWRITEBYTECODE=1', + '-v', `${root}:/work:ro`, '-w', '/work', '--entrypoint', 'python3', TEST_IMAGE, ...cmd]; +const containerReady = () => { + if (!have('docker')) return 'docker not available'; + if (sh('docker', ['image', 'inspect', TEST_IMAGE]).status) return `image ${TEST_IMAGE} missing; build: docker build -f docker/Dockerfile.test -t ${TEST_IMAGE} docker`; + return ''; +}; const pyMod = (m) => sh(PY, ['-m', m, '--version']).status === 0; const tail = (r, n = 15) => ((r.stdout || '') + (r.stderr || '')).trim().split('\n').slice(-n).join('\n'); // A stage returns {status:'PASS'|'FAIL'|'SKIPPED', note} @@ -60,6 +71,11 @@ const stages = [ }], ['types', 'type check', () => skip('no type checker configured')], ['pyunit', 'python unittest', () => { + if (container) { + const why = containerReady(); + if (why) return skip(`--container: ${why}`); + return result(inherit('docker', containerArgs('-m', 'unittest', 'discover', '-s', 'tests', '-p', 'test_*.py')), 'throwaway container (Python 3.11); see output above'); + } const { env, cleanup } = isolatedPyEnv(); try { return result(inherit(PY, ['-m', 'unittest', 'discover', '-s', 'tests', '-p', 'test_*.py'], { env }), 'see unittest output above'); } finally { cleanup(); }