tts-voice-creator-clone-and-design-2/architecture/qa.md

# QA: `npm run qa`

Developer-facing. Single entry point: `scripts/qa.mjs` (Node, no dependencies). Every stage is timed; a summary table is printed and the exit code is non-zero if any stage FAILs. SKIPPED never fails the run. Nothing is ever installed automatically.

```bash
npm run qa -- --fast        # everything except Playwright
npm run qa                  # full, incl. Playwright (desktop + iphone)
npm run qa -- --only e2e --grep voice-provenance
npm run qa -- --sonar       # add the SonarQube stage
npm run qa -- --list
```

## Stages

| id | what | skipped when |
|----|------|--------------|
| format | formatter check | always for now (no formatter configured) |
| lint | `node --check` over tracked static/js, scripts, tests mjs; `npm run check:frontend`; python `compileall`; `ruff` if installed | ruff missing -> noted only |
| types | type check | no type checker configured |
| pyunit | `unittest discover -s tests -p 'test_*.py'` with a temp `HOME`/`CONFIG_DIR` (avoids the `/home/app` permission error, never touches live data) | never |
| nodeunit | `npm run test:unit` | never |
| e2e | Playwright, projects `desktop` + `iphone`; `--grep` supported | `--fast` |
| a11y | axe-core specs | axe-core not installed (recommend `npm i -D @axe-core/playwright`, add `tests/a11y.spec.js`) |
| security | `npm audit --audit-level=high`, `pip-audit` (if installed + requirements.txt), secret scan (gitleaks if present, else built-in regexes over tracked files) | audits: network failure -> SKIPPED, not FAIL |
| build | `npm run minify`, fails if `static/dist/main.min.js` changed (stale bundle; commit the regenerated file) | never |
| perf | DOM budget is `tests/constrained.spec.js`, runs inside e2e | always (no separate stage) |
| sonar | `sonar-scanner` with `sonar-project.properties` | no `--sonar`, or `SONAR_HOST_URL`/`SONAR_TOKEN` unset, or scanner missing |

## Test isolation

Playwright `baseURL` defaults to the live container (`http://localhost:7890`, override with `BASE_URL`). The specs mock every `/api/**` route (the live server only serves static files and the page), so no live data is written; use a throwaway instance via `BASE_URL` when in doubt.

**Rule: specs must never write to the live API.** Every spec imports `test`/`expect` from `tests/_guard.js` (not from `@playwright/test`). Its auto fixture registers `page.route('**/api/**')` first; Playwright consults the most recently registered route first, so spec mocks win and only unmocked requests reach the guard. Any non-GET/HEAD/OPTIONS request that reaches it is aborted and the test fails at teardown naming `METHOD url`. Fix by adding a `page.route` mock in the spec, never by weakening the guard. Unmocked GETs are tolerated. Default inert mocks for app-init writes (design-preset sync, voice-source scrape) live in `_guard.js` and lose to any spec route. Limit: `request`/`page.request` (APIRequestContext) bypass page routes and are not covered. `--fast` never touches it. The secret scan prints only `file:line pattern-name`, never the match.

## Isolation and coverage

- `pyunit` and the coverage run each get a fresh `mkdtemp` HOME/CONFIG_DIR/TMPDIR (`isolatedPyEnv` in `scripts/qa.mjs`), removed afterwards. Stages run strictly sequentially, so e2e and pyunit never overlap.
- Sonar stage: if `python3 -c 'import coverage'` works, it runs `coverage run -m unittest discover ...` and writes `coverage.xml` (read via `sonar.python.coverage.reportPaths`). Otherwise it prints SKIPPED; install with `pip install coverage`.
- JS coverage is NOT collected. Playwright-driven coverage needs c8/V8 instrumentation of the served bundle (e.g. `page.coverage` + `v8-to-istanbul` -> `sonar.javascript.lcov.reportPaths`). Owner decision pending.

## SonarQube / secrets

`SONAR_HOST_URL` and `SONAR_TOKEN` are read from the environment only (shell or CI secrets). They are never written to files, argv or output (the scanner reads the token itself; output is redacted). The quality gate is awaited via `-Dsonar.qualitygate.wait=true`; no separate web API call is made.

## CI

`.github/workflows/sonarqube.yml` already runs compile/unittest/frontend/build steps and the Sonar action (token from `${{ secrets.SONAR_TOKEN }}`). An advisory (`continue-on-error`) `npm run qa -- --fast` step was added. Make it blocking once the pre-existing failures below are fixed.

## Known pre-existing failures (2026-10-04)

- `check:frontend`: duplicate top-level `say` in `static/js/voice-archetypes.js` and `static/js/magazine.js`.
- `test_epub_export.EndpointTests.test_downloads_a_valid_epub`.
- Secret scan: tracked `scratch/test_openai.py` and `scratch/test_google_imagen.py` contain key-like literals; rotate if real and remove from history/tracking.

## Python interpreter
`scripts/qa.mjs` runs Python through the first interpreter that can `import fastapi`: `$PYTHON`, `python3` on PATH, `~/miniforge3/bin/python3`, `/usr/bin/python3`. Agents started by the multica daemon often see `/usr/bin/python3` first (no fastapi), which made their unit tests fail on import; `npm run qa` now works there too.

## Python tests in a throwaway container
Run: `npm run qa -- --only pyunit --container` (or `QA_PY_CONTAINER=1`). Default stays the host interpreter.

Why: production runs Python 3.11 with the dependencies installed from `requirements.txt` at image build time (unpinned, so newer than a host's: e.g. starlette 1.7 in the image vs 1.3 on miniforge). The container run tests that exact combination and never touches the live service.

Setup (once, local only, nothing pushed): `docker build -f docker/Dockerfile.test -t tts-voice-creator-test:local docker`. `docker/Dockerfile.test` is `FROM` the app image plus `httpx2` (needed by `starlette.testclient`, not shipped in production). Without docker or the image the stage reports SKIPPED with the build command.

Container: `docker run --rm --network none --read-only --tmpfs /tmp:rw,exec --tmpfs /home/app:rw`, `HOME`/`CONFIG_DIR`/`TMPDIR` under `/tmp`, user `app`. The repo is mounted read-only at `/src`; the tests run in a writable throwaway copy at `/tmp/work` (tar copy without `.git`, `node_modules`, `config`, `exports`, `logs`, `attachments`, `.multica`, the nested project dir; `config/regression_snapshots` is added back because a test reads it). Some tests write to the tree (e.g. `os.utime`), so the original is never written.

The test image also has `nodejs` (Debian package, aarch64) for the browser/server fingerprint test, and `httpx2`.

Result: 1086 tests, OK, 3 skipped in the container (host: 1086 OK). Skips: `test_guided_installer.RunCommandTest` (3 tests), reason 'tkinter unavailable' (the image has no tkinter). No Python 3.11 vs 3.12 behaviour differences found. Coverage (`--sonar`) still uses the host interpreter.