tts-voice-creator-clone-and-design-2/architecture/character-visual-identity.md

# SOP — Character visual identity and appearance forms

Status: implemented. Covers how a character's avatar and concept (reference) sheet stay the same
person, and how one character holds several appearances ("forms") without splitting into several
characters or voices.

## Single source of truth

* Appearance facts come from the sheet's `physical`, `clothing`, `age_estimate`, `race_species`,
  `gender`, `title`, `profession`, `archetype` (and the user overrides `visual_appearance`,
  `visual_species`, `visual_costume`). Both prompts are built by
  `_character_visual_facts` in `routes/conversation.py`; the active form is overlaid first
  (`_character_form_sheet`, which never lets a form change its own `id`/`name`).
* `image_prompt` (portrait) and `concept_art_prompt` (reference sheet) are **deterministic** —
  built server-side from those facts with fixed camera/lighting/period text. Sheet extraction no
  longer asks the LLM to write `image_prompt` (it drifted from the concept sheet); the prompt is
  generated right after extraction by `csAutoGenerateExternalPrompts`, and again before any
  avatar-only or per-form generation. A hand-edited prompt (`*_user_edited`) is never overwritten.
* Only the two prompts describe an appearance. `voice_design_prompt` and the SillyTavern prompt
  belong to the character as a whole, so they always live on the base sheet.

## Stale detection

`_character_identity_fingerprint` (server) and `_libIdentityFingerprint` (browser) hash the same
field list; a test runs the browser code in Node and compares, including list values and non-BMP
characters. The three `visual_*` overrides are appended only when set, so pairs made before they
existed keep a valid fingerprint. A pair needs review when its prompt version is not current, its
fingerprint is missing or differs, or `identity_consistent` is false. Regenerating **only** the
avatar or **only** the reference image sets `identity_consistent = false` on an existing pair,
because a lone regeneration produces a new face.

## Where the data lives

| | base appearance | a form |
|---|---|---|
| profile image | record `image` | `form.image` |
| reference image, version, fingerprint, digests, `identity_consistent` | `sheet.*` | `form.*` |
| Physical / Clothing text | `sheet.physical/clothing` | `form.physical/clothing` |

All visual results are written through one function (`_libSaveCharacterVisual`), which re-reads the
stored record, writes to the active form or the base, and saves. (It used to write the base
reference image to the record root, and the consistent-pair generator kept it in memory only, so a
base pair lost its reference image and metadata on reload.)

`GET /api/characters/visual-pair-review` checks the base pair and every form that has images, each
against its own metadata and its own fingerprint, and reports form pairs under the form name.

## Forms (max 8 per character, enforced in `core/validation.py`)

Add (starts as a copy of the base Physical/Clothing; only non-empty keys are stored because an
empty key would blank the base value when overlaid), rename (unique, ≤160 chars), delete (confirm;
the character, its voice and the base look stay), show, and generate matched images for one form or
for all forms. Generation rebuilds that form's prompts first (unless hand-edited), saves the pair on
that form, then restores whichever appearance was active. Editing Physical/Clothing while a form is
active changes only that form.


## Image prompt hygiene and the turnaround (1.34.x)

How sheets are extracted and merged, which identity tokens count, the story-moment quote rule and the
age rules: see `character-extraction.md`.

* **Setting.** Prompts describe the books as *The Dark Eye* high-fantasy medieval dress (about 900-1300 AD: wool and linen tunics, belted robes, leather, mail where the sheet names it). The earlier "early Renaissance 1420-1500" wording pulled uniforms, coats, hats and boots that read as musketeer/cowboy/wartime.
* **Wording.** Extracted costume text is rewritten at prompt time (`_period_wording`: Uniform, Weste, Hut, Sporen, Jacke, Schal … become surcoat, jerkin, hood, boots, tunic …). Stored sheets are not changed.
* **Injuries and dirt.** `_strip_injury_clauses` removes clauses about blood, wounds, bandages, burns, limping, torn clothes, soot and dirt from appearance, costume and the story moment; the rest of the sentence stays. The portrait baseline asks for clean skin and garments.
* **Variety.** Characters with armour or weapons get an individual garment cut and colour scheme (`_character_design_variation`, stable per name; colours the sheet names win) instead of one shared list.
* **The turnaround stays one T2I triptych.** Drawing front/side/rear one by one with the edit model was tried (2026-09-20) and reverted the next day: it returned three near-identical front views, mixed rendering styles between panels, cropped bodies, and copied the wireframe pose guide into the costume — exactly the failure the triptych was introduced to fix. The portrait/sheet face mismatch is still open and needs a different approach.
* **Props** get an object-only period sentence, so a tile no longer turns into a portrait of the character.
* Older saved prompts are cleaned the same way at render time. Existing image pairs are unchanged until regenerated.


## Automatic image check (`POST /api/character-image-check`)

A vision model reports **only what it sees** — figure count, the view of each figure left to right,
whether every head and foot is in frame, whether the figures are the same person, apparent gender,
ear shape (pointed/round/hidden), wounds, and background (plain/scenery). The expected values come from
the character sheet and the comparison happens in code (`core/character_image_check.py`). Asking a model
"is this a woman, as required?" invites agreement; asking "what gender is this?" does not, and the
expected gender is never revealed in the question.

* Kinds: `portrait`, `turnaround`, `scene`, `same_person` (profile vs. a sheet figure).
* **Fail-safe:** an unreachable checker, an unreadable answer, or a missing observation is a failure,
  never a pass. An unreadable answer is returned verbatim (truncated) so it can be diagnosed.
* Thinking is switched off: a reasoning model spent the whole budget deliberating over a confusing image
  and returned nothing.
* **Model:** the active LLM by default (`llm_model`), overridable with the `image_check_model` setting.
  The model already loaded reads images, so a check never makes llama-swap unload it for a separate
  vision model on a GPU with no room for both.

**Calibration (2026-09-21)** on seven panels whose defects were known (three front views, elf ears on a
human, a wireframe rendered as a figure, cut-off feet, a man where the sheet says woman, a forest behind a
studio portrait, and one correct same-person pair), run twice:

| model | correct | wrong | unreadable | per check |
|---|---|---|---|---|
| local Qwen3.8-27B (`default`) | 14/14 | 0 | 0 | ~4 s |
| Ling 3.0 Flash VL (`vision`, free, OpenRouter) | 8/14 | 0 | 6 | ~17 s |

Ling never judged wrongly but on every turnaround ignored the JSON-only instruction and answered in prose.
It also has free-tier rate limits and sends the images to a third party.


## Generation with checks and retries (face from the profile)

The generation job (`POST /api/character-generate-consistent-images`) now runs:

1. **Identity image** (text-to-image, full body). The profile picture is cut from it. **Portrait check**
   on exactly that crop (one person, gender, ears, wounds, plain background). On failure: a new seed,
   up to `image_check_max_attempts` (default 3), keeping the attempt with the fewest faults.
2. **Sheet** (turnaround triptych + story moment + props, one ComfyUI run). The profile's face is then
   transferred with **ReActor** (`inswapper_128`, CodeFormer 0.5 restore): turnaround faces left to right
   (front and side; the rear has none), and in the story panel only the *largest* face — the character in
   the foreground, not an extra. **Checks:** turnaround, story panel, and same-person (profile vs. the front
   view). Gender is judged *after* the face transfer, on what a viewer actually sees. Retried the same way.
3. The result carries `image_check` = `{verified, passed, attempts, failures[], checks{}}`, saved on the
   sheet and shown on the character page: verified with no faults / faults still found (listed) / not
   verified.

**If the checker itself is unavailable, retries stop at once** and the pair is marked *not verified*:
re-rolling images cannot fix a missing checker, and would burn up to three renders per character.

Why face transfer and not rendering the views from the portrait: the edit model keeps the source pose
(three front views, the pose guide copied into the costume — tried and reverted 2026-09-21). The
text-to-image triptych gets the poses right; swapping only the face afterwards fixes the identity.

Settings: `image_check_enabled` (default on), `image_check_max_attempts` (1–5, default 3),
`image_check_model` (default: the active LLM).


## The turnaround prompt is short and framing-first (2026-09-21)

Measured with turnaround-only renders scored by the image check (`scratchpad feet_experiment.py`):

| turnaround prompt | full body, head to shoes | plain backdrop |
|---|---|---|
| long compact prompt (~3 000 chars, framing rules mid-prompt), 1536x1024 | 0/5 | 2/5 |
| same, taller 1536x1280 canvas | 2/5 | 0/5 |
| same + "visible floor, contact shadow" wording | 0/5 | — |
| **short, framing first (~770 chars)**, 1536x1024 and 1536x1280 | **10/10** | **10/10** |
| the app's new prompt on Karyla, Darrag, Eolan (4 seeds each, after the fixes below) | **12/12** | **12/12** |

The 4-step Qwen-Image model follows the start of a prompt and loses the tail, so wording changes inside the
long prompt did nothing. The prompt now opens with the framing (three full-body views side by side,
head to soles, empty space above and below, plain white studio), then the character (subject capped at
700 chars), then only the rules a sheet needs, one short sentence each: leather-only armour, headwear,
elf ears (only for elves), child/teen proportions, clean skin and clean well-kept clothes.

What the validation runs found, each fixed and pinned by a test:
* **Passing states drawn as features** — "rote Stirn bei Wut, Tränen … vor Wut" became red marks (Darrag);
  "Zunge streicht über die Lippen" a tongue sticking out (Eolan). Anger, tears, trembling, twitching,
  blushing, sweating, licking are stripped like wounds.
* **Scars** — even a scar the sheet calls almost faded, with "fully healed, pale silvery line" added,
  rendered as a fresh red gash, and that sentence put slashes on hands and robe. Scars are left out of
  image prompts entirely; this model cannot draw a subtle one.
* **A dropped clause starting a sentence** left fragments ("und die rechte Augenbraue"): the whole sentence
  now goes. "faded blue wool coat" stays (a costume, not a scar).
* Hedges "unklar" / "wird erwähnt" drop speculative sentences.

**Portrait (identity image) got the same treatment.** Old prompt ~2 800-3 700 characters, new ~875, framing
first ("full-length studio photograph … head to shoes … head in the upper third … plain white studio"). Identity
renders cropped exactly as the profile picture is, scored by the portrait check, 4 seeds each:

| | old prompt | new prompt |
|---|---|---|
| Karyla | 0/4 (scenery backdrop every time) | 4/4 |
| Darrag | 4/4 | 4/4 |

Both prompts share one rules line, which now again carries the per-character costume cut and colours for armed
characters (`_character_design_variation`; the first short turnaround had dropped it) and names the setting
("in the world of The Dark Eye"). The portrait asks for empty hands; weapon handling stays in the story prompt.

**The sheet follows the finished portrait.** After the portrait passes its check, the vision model describes what
it shows (`look`: hair colour/length/style, facial hair, eye colour, `look_phrase`, punctuation stripped and short),
and the turnaround and story prompts are rebuilt with "exactly as in the portrait: …". Found on Darrag, whose sheet
says nothing about hair: the portrait had shoulder-length hair and the turnaround a crop; the face transfer copied
the face but not the hair, and the same-person check failed on all three sheet attempts. Best effort: no
description, no change.

Open: robes can still come out with ragged hems; the story-moment prompt is still long.


## English prompt inputs (2026-10-03)

Sheets are extracted from German books, so their free text is mostly German. Image models do best with English,
and German leaked into keyword logic ("hat dünne Arme", "has thin arms", became a hood; "Mensch (oder Zwerg …)"
confused species checks). Every image-prompt builder therefore reads an **English copy** of the free text.

* **Layer:** `core/prompt_translation.py` (no network/DB code; the LLM is injected). Wiring:
  `_english_prompt_inputs` in `routes/conversation.py`, called by `/api/character-generate-prompts` and
  `/api/character-generate-consistent-images` *before* the builders and before species/age/dwarf/period logic run.
* **Fields:** species, appearance, clothing, age, title, profession, archetype, hair/eyes/build/marks, era,
  mannerisms/capabilities, props/inventory, visual overrides, the chosen story-moment quote (as `visual_scene`)
  and the book profile (genre, setting, era, visual context). Names and aliases are never translated.
* **Detection:** cheap stop-word/vocabulary heuristic; English or empty text is skipped without an LLM call.
* **Translation:** one batched JSON call per character through the app's existing LLM endpoint (`llm_url`,
  model `prompt_translation_model` or `llm_model`; same retry/backoff helper), temperature 0.1, strict prompt
  (faithful, keep names, no added or dropped traits, JSON only).
* **Validation (per field):** non-empty, not an error/refusal, length sanity, not an unchanged German echo,
  not still German, names present in the output when present in the source, list shape kept. A rejected field
  keeps its original text; the other fields still use their translation.
* **Cache:** table `prompt_translation_cache` in the existing SQLite DB (derived data, separate from
  `characters`), key = `sha256(source)|lang|model|prompt version`. Editing the source text, switching the
  model or bumping `PROMPT_VERSION` misses automatically. Records are never rewritten; the UI keeps showing
  the German source of truth. The identity fingerprint and the image checks keep using the original sheet.
* **Fallback:** LLM down, invalid output or flag off: the original text is used (the German keyword rules
  stay as the fallback), a warning is logged, and the response carries `prompt_translation`
  (`used_untranslated_text`, `translated_fields`, `untranslated_fields`, `cache_hits`, `error`). Image
  generation is never blocked.
* **Flag:** env `PROMPT_TRANSLATION=off` (or settings key `prompt_translation: false`); default on.
* **Dry run:** `POST /api/character-prompt-preview` with `{id}` or `{name, book, sheet, book_profile}` returns the
  prompts built from the original and from the English text plus the changed fields; it generates nothing and
  writes no record (only the translation cache).
* **Side fixes:** `_character_age_category` now reads `age_estimate` (sheets store that key; a boy used to render
  adult); `_period_wording` no longer turns German "hat" (has) into a hood.


## Distinct faces per character (`core/face_profile.py`, 2026-10-03)

**Problem (owner report):** the faces of different characters of one book look like one family. Measured on the
real library (read-only snapshot, 4 books): 28-68 of 49-83 sheets name **no** facial feature at all, so the
prompt carried none either, and the image model (Qwen-Image 2512 + 4-step Lightning LoRA, cfg 1.0 in every bundled
graph) answers with its own default face. Seeds are already per prompt/character (`voice-creator-image-v5|<prompt>`,
`voice-creator-cast-v6|<book>|<style>|<name>|<uuid>`), so seeds are not the cause. Generic wording made it worse:
"one fixed studio artist, with consistent facial construction" (concept style), "a full, thick head of hair
reaching at least to the ears or collar, simply combed, tied back, or left loose", "Long hair reaching past the
shoulders" (armoured women) and "a thick, full beard" in the dwarf body lock (also for dwarf women).

**Design.** A deterministic, explainable *face profile* gives every humanoid character one value per slot:
face shape, jaw/chin, nose, eye shape, eye colour, brows, mouth, cheekbones, skin, hair colour, hair style,
facial hair, age signs, one distinguishing mark. Pure functions, no I/O, no randomness; the clause is the
compact text `face: oval face, narrow pointed chin, ...` that goes into the Subject right after species (high
priority; a hand-edited Subject without it gets it prepended).

1. **Sheet first.** Traits the sheet states (German or English: `physical` / `visual_appearance`) are read by
   keyword rules and kept (`source: sheet`). Stated hair length/texture is kept as the sheet's own words.
   Scars and tattoos occupy the mark slot but are never written into the prompt (existing hygiene: the model draws
   a scar as a wound). Unknown wording is simply not read: nothing is invented that contradicts what was read.
2. **Owner override.** `visual_face` (free text on the sheet) replaces the generated clause verbatim; it is still
   parsed so the character keeps its place in the distinctness assignment.
3. **Pools, filtered.** Each option carries species (human/elf/dwarf/orc/vampire, half-elf = human+elf), age stage
   (child, teen, young, middle, old), gender and a culture exclusion. Facial hair exists only for men from
   young adulthood, never for elves; old people get grey/white hair and old-age skin; elves show no age; vampires
   stay pale; a culture named in `nationality_background` (north/south) removes implausible skin options only.
   Unknown gender gets only styles that read either way. Species is the character's own leading assertion
   (`Mensch (oder Zwerg ...)` is human); a raven, golem, narrator etc. gets no profile. Ears stay with
   `_character_ear_shape`; the clause never mentions ears.
4. **Spread over the cast.** `char_face_roster(book)` reads the cast in creation order (a light SQL extract, no
   images). Each character takes, per free slot, the value that collides least with the earlier characters,
   weighted by how similar the colliding character already is (greedy max-distance); ties by
   `sha256(book|name|slot|option)`. A character depends only on those before it, so **adding characters never
   changes an existing profile** (a new, unsaved character goes last). Deleting an earlier character, or editing
   a sheet's traits, can shift later ones; pin a face with `visual_face` where that matters.
5. **Relatives** named in `relationships` next to a kinship word (brother, Vater: ...) pass on family slots
   (eye colour, hair colour, nose, jaw, each by a hash coin flip) and are ignored in the collision cost; all other
   slots still avoid each other.
6. **Hair locks follow the profile.** When the clause states hair, the "reaching to the ears or collar" and
   "past the shoulders" wording is replaced by "the hair described in the face clause". The dwarf body lock no
   longer carries a beard; the beard (men only) comes from the clause.

**Not added:** a "not the same face as other characters" negative. The bundled graphs run cfg 1.0, where the
negative conditioning is skipped, and the positive encoder draws any noun it reads.

**Distinctness measure.** `distance(a, b)` = weighted share of slots that differ (hair 1.5, face shape/skin/nose/beard
1.2, others 0.8-1.0). *Before* = what the old prompt carried: only slots the sheet states; every other slot is the
"model default" and counts as equal. *After* = the profile. This is a descriptor-space proxy, not a measurement of
rendered images.

**Dry run (generates nothing, writes nothing):** `POST /api/character-face-preview {book}` returns each
character's profile (slot, text, source) and the before/after mean pairwise distance, the most similar pairs and
the similarity matrix; `POST /api/character-prompt-preview` now also returns `face_clause`. Prompts get the clause
through `_attach_face_profile` in `character_generate_prompts`, `character_generate_consistent_images` and the
preview; without cast context `_character_visual_facts` falls back to the character's own solo profile.

**Limits.** Existing images are unchanged until regenerated, and the identity fingerprint is unchanged (the
profile is derived, not stored), so pairs are not flagged stale. The model may still blur details at cfg 1.0 with
4 steps; if regenerated faces stay too similar the options are a higher-cfg/more-steps pass for the portrait
(slower), or a per-character face LoRA/IP-adapter reference. Visual verification needs real regenerations.