tts-voice-creator-clone-and-design-2/architecture/qa.md

QA: npm run qa

Developer-facing. Single entry point: scripts/qa.mjs (Node, no dependencies). Every stage is timed; a summary table is printed and the exit code is non-zero if any stage FAILs. SKIPPED never fails the run. Nothing is ever installed automatically.

npm run qa -- --fast        # everything except Playwright
npm run qa                  # full, incl. Playwright (desktop + iphone)
npm run qa -- --only e2e --grep voice-provenance
npm run qa -- --sonar       # add the SonarQube stage
npm run qa -- --list

Stages

id	what	skipped when
format	formatter check	always for now (no formatter configured)
lint	node --check over tracked static/js, scripts, tests mjs; npm run check:frontend; python compileall; ruff if installed	ruff missing -> noted only
types	type check	no type checker configured
pyunit	unittest discover -s tests -p 'test_*.py' with a temp HOME/CONFIG_DIR (avoids the /home/app permission error, never touches live data)	never
nodeunit	npm run test:unit	never
e2e	Playwright, projects desktop + iphone; --grep supported	--fast
a11y	axe-core specs	axe-core not installed (recommend npm i -D @axe-core/playwright, add tests/a11y.spec.js)
security	npm audit --audit-level=high, pip-audit (if installed + requirements.txt), secret scan (gitleaks if present, else built-in regexes over tracked files)	audits: network failure -> SKIPPED, not FAIL
build	npm run minify, fails if static/dist/main.min.js changed (stale bundle; commit the regenerated file)	never
perf	DOM budget is tests/constrained.spec.js, runs inside e2e	always (no separate stage)
sonar	sonar-scanner with sonar-project.properties	no --sonar, or SONAR_HOST_URL/SONAR_TOKEN unset, or scanner missing

Test isolation

Playwright baseURL defaults to the live container (http://localhost:7890, override with BASE_URL). The specs mock every /api/** route (the live server only serves static files and the page), so no live data is written; use a throwaway instance via BASE_URL when in doubt.

Rule: specs must never write to the live API. Every spec imports test/expect from tests/_guard.js (not from @playwright/test). Its auto fixture registers page.route('**/api/**') first; Playwright consults the most recently registered route first, so spec mocks win and only unmocked requests reach the guard. Any non-GET/HEAD/OPTIONS request that reaches it is aborted and the test fails at teardown naming METHOD url. Fix by adding a page.route mock in the spec, never by weakening the guard. Unmocked GETs are tolerated. Default inert mocks for app-init writes (design-preset sync, voice-source scrape) live in _guard.js and lose to any spec route. Limit: request/page.request (APIRequestContext) bypass page routes and are not covered. --fast never touches it. The secret scan prints only file:line pattern-name, never the match.

Isolation and coverage

• pyunit and the coverage run each get a fresh mkdtemp HOME/CONFIG_DIR/TMPDIR (isolatedPyEnv in scripts/qa.mjs), removed afterwards. Stages run strictly sequentially, so e2e and pyunit never overlap.
• Sonar stage: if python3 -c 'import coverage' works, it runs coverage run -m unittest discover ... and writes coverage.xml (read via sonar.python.coverage.reportPaths). Otherwise it prints SKIPPED; install with pip install coverage.
• JS coverage is NOT collected. Playwright-driven coverage needs c8/V8 instrumentation of the served bundle (e.g. page.coverage + v8-to-istanbul -> sonar.javascript.lcov.reportPaths). Owner decision pending.

SonarQube / secrets

SONAR_HOST_URL and SONAR_TOKEN are read from the environment only (shell or CI secrets). They are never written to files, argv or output (the scanner reads the token itself; output is redacted). The quality gate is awaited via -Dsonar.qualitygate.wait=true; no separate web API call is made.

CI

.github/workflows/sonarqube.yml already runs compile/unittest/frontend/build steps and the Sonar action (token from ${{ secrets.SONAR_TOKEN }}). An advisory (continue-on-error) npm run qa -- --fast step was added. Make it blocking once the pre-existing failures below are fixed.

Known pre-existing failures (2026-10-04)

• check:frontend: duplicate top-level say in static/js/voice-archetypes.js and static/js/magazine.js.
• test_epub_export.EndpointTests.test_downloads_a_valid_epub.
• Secret scan: tracked scratch/test_openai.py and scratch/test_google_imagen.py contain key-like literals; rotate if real and remove from history/tracking.

Python interpreter

scripts/qa.mjs runs Python through the first interpreter that can import fastapi: $PYTHON, python3 on PATH, ~/miniforge3/bin/python3, /usr/bin/python3. Agents started by the multica daemon often see /usr/bin/python3 first (no fastapi), which made their unit tests fail on import; npm run qa now works there too.

Python tests in a throwaway container

Run: npm run qa -- --only pyunit --container (or QA_PY_CONTAINER=1). Default stays the host interpreter.

Why: production runs Python 3.11 with the dependencies installed from requirements.txt at image build time (unpinned, so newer than a host's: e.g. starlette 1.7 in the image vs 1.3 on miniforge). The container run tests that exact combination and never touches the live service.

Setup (once, local only, nothing pushed): docker build -f docker/Dockerfile.test -t tts-voice-creator-test:local docker. docker/Dockerfile.test is FROM the app image plus httpx2 (needed by starlette.testclient, not shipped in production). Without docker or the image the stage reports SKIPPED with the build command.

Container: docker run --rm --network none --read-only --tmpfs /tmp:rw,exec --tmpfs /home/app:rw, HOME/CONFIG_DIR/TMPDIR under /tmp, repo mounted read-only at /work, user app, entrypoint python3.

Known differences from the host run (1086 tests, all OK on host; container: 2 errors, 3 skipped):

• test_asset_versions...build_id_changes_when_a_script_changes: the test calls os.utime on a repo file; fails on the read-only mount (the mount is deliberate).
• test_character_identity...browser_and_server_compute_identical_fingerprints: needs node, which the app image does not contain. Neither is a Python 3.11 vs 3.12 behaviour difference; the other tests pass on 3.11. Coverage (--sonar) still uses the host interpreter.