tts-voice-creator-clone-and-design-2/architecture/character-visual-identity.md

SOP — Character visual identity and appearance forms

Status: implemented. Covers how a character's avatar and concept (reference) sheet stay the same person, and how one character holds several appearances ("forms") without splitting into several characters or voices.

Single source of truth

• Appearance facts come from the sheet's physical, clothing, age_estimate, race_species, gender, title, profession, archetype (and the user overrides visual_appearance, visual_species, visual_costume). Both prompts are built by _character_visual_facts in routes/conversation.py; the active form is overlaid first (_character_form_sheet, which never lets a form change its own id/name).
• image_prompt (portrait) and concept_art_prompt (reference sheet) are deterministic — built server-side from those facts with fixed camera/lighting/period text. Sheet extraction no longer asks the LLM to write image_prompt (it drifted from the concept sheet); the prompt is generated right after extraction by csAutoGenerateExternalPrompts, and again before any avatar-only or per-form generation. A hand-edited prompt (*_user_edited) is never overwritten.
• Only the two prompts describe an appearance. voice_design_prompt and the SillyTavern prompt belong to the character as a whole, so they always live on the base sheet.

Stale detection

_character_identity_fingerprint (server) and _libIdentityFingerprint (browser) hash the same field list; a test runs the browser code in Node and compares, including list values and non-BMP characters. The three visual_* overrides are appended only when set, so pairs made before they existed keep a valid fingerprint. A pair needs review when its prompt version is not current, its fingerprint is missing or differs, or identity_consistent is false. Regenerating only the avatar or only the reference image sets identity_consistent = false on an existing pair, because a lone regeneration produces a new face.

Where the data lives

	base appearance	a form
profile image	record image	form.image
reference image, version, fingerprint, digests, identity_consistent	sheet.*	form.*
Physical / Clothing text	sheet.physical/clothing	form.physical/clothing

All visual results are written through one function (_libSaveCharacterVisual), which re-reads the stored record, writes to the active form or the base, and saves. (It used to write the base reference image to the record root, and the consistent-pair generator kept it in memory only, so a base pair lost its reference image and metadata on reload.)

GET /api/characters/visual-pair-review checks the base pair and every form that has images, each against its own metadata and its own fingerprint, and reports form pairs under the form name.

Forms (max 8 per character, enforced in core/validation.py)

Add (starts as a copy of the base Physical/Clothing; only non-empty keys are stored because an empty key would blank the base value when overlaid), rename (unique, ≤160 chars), delete (confirm; the character, its voice and the base look stay), show, and generate matched images for one form or for all forms. Generation rebuilds that form's prompts first (unless hand-edited), saves the pair on that form, then restores whichever appearance was active. Editing Physical/Clothing while a form is active changes only that form.

Image prompt hygiene and the turnaround (1.34.x)

• Setting. Prompts describe the books as The Dark Eye high-fantasy medieval dress (about 900-1300 AD: wool and linen tunics, belted robes, leather, mail where the sheet names it). The earlier "early Renaissance 1420-1500" wording pulled uniforms, coats, hats and boots that read as musketeer/cowboy/wartime.
• Wording. Extracted costume text is rewritten at prompt time (_period_wording: Uniform, Weste, Hut, Sporen, Jacke, Schal … become surcoat, jerkin, hood, boots, tunic …). Stored sheets are not changed.
• Injuries and dirt. _strip_injury_clauses removes clauses about blood, wounds, bandages, burns, limping, torn clothes, soot and dirt from appearance, costume and the story moment; the rest of the sentence stays. The portrait baseline asks for clean skin and garments.
• Variety. Characters with armour or weapons get an individual garment cut and colour scheme (_character_design_variation, stable per name; colours the sheet names win) instead of one shared list.
• The turnaround stays one T2I triptych. Drawing front/side/rear one by one with the edit model was tried (2026-09-20) and reverted the next day: it returned three near-identical front views, mixed rendering styles between panels, cropped bodies, and copied the wireframe pose guide into the costume — exactly the failure the triptych was introduced to fix. The portrait/sheet face mismatch is still open and needs a different approach.
• Props get an object-only period sentence, so a tile no longer turns into a portrait of the character.
• Older saved prompts are cleaned the same way at render time. Existing image pairs are unchanged until regenerated.

Automatic image check (POST /api/character-image-check)

A vision model reports only what it sees — figure count, the view of each figure left to right, whether every head and foot is in frame, whether the figures are the same person, apparent gender, ear shape (pointed/round/hidden), wounds, and background (plain/scenery). The expected values come from the character sheet and the comparison happens in code (core/character_image_check.py). Asking a model "is this a woman, as required?" invites agreement; asking "what gender is this?" does not, and the expected gender is never revealed in the question.

• Kinds: portrait, turnaround, scene, same_person (profile vs. a sheet figure).
• Fail-safe: an unreachable checker, an unreadable answer, or a missing observation is a failure, never a pass. An unreadable answer is returned verbatim (truncated) so it can be diagnosed.
• Thinking is switched off: a reasoning model spent the whole budget deliberating over a confusing image and returned nothing.
• Model: the active LLM by default (llm_model), overridable with the image_check_model setting. The model already loaded reads images, so a check never makes llama-swap unload it for a separate vision model on a GPU with no room for both.

Calibration (2026-09-21) on seven panels whose defects were known (three front views, elf ears on a human, a wireframe rendered as a figure, cut-off feet, a man where the sheet says woman, a forest behind a studio portrait, and one correct same-person pair), run twice:

model	correct	wrong	unreadable	per check
local Qwen3.8-27B (default)	14/14	0	0	~4 s
Ling 3.0 Flash VL (vision, free, OpenRouter)	8/14	0	6	~17 s

Ling never judged wrongly but on every turnaround ignored the JSON-only instruction and answered in prose. It also has free-tier rate limits and sends the images to a third party.

Generation with checks and retries (face from the profile)

The generation job (POST /api/character-generate-consistent-images) now runs:

1. Identity image (text-to-image, full body). The profile picture is cut from it. Portrait check on exactly that crop (one person, gender, ears, wounds, plain background). On failure: a new seed, up to image_check_max_attempts (default 3), keeping the attempt with the fewest faults.
2. Sheet (turnaround triptych + story moment + props, one ComfyUI run). The profile's face is then transferred with ReActor (inswapper_128, CodeFormer 0.5 restore): turnaround faces left to right (front and side; the rear has none), and in the story panel only the largest face — the character in the foreground, not an extra. Checks: turnaround, story panel, and same-person (profile vs. the front view). Gender is judged after the face transfer, on what a viewer actually sees. Retried the same way.
3. The result carries image_check = {verified, passed, attempts, failures[], checks{}}, saved on the sheet and shown on the character page: verified with no faults / faults still found (listed) / not verified.

If the checker itself is unavailable, retries stop at once and the pair is marked not verified: re-rolling images cannot fix a missing checker, and would burn up to three renders per character.

Why face transfer and not rendering the views from the portrait: the edit model keeps the source pose (three front views, the pose guide copied into the costume — tried and reverted 2026-09-21). The text-to-image triptych gets the poses right; swapping only the face afterwards fixes the identity.

Settings: image_check_enabled (default on), image_check_max_attempts (1–5, default 3), image_check_model (default: the active LLM).