212 lines
15 KiB
Markdown
212 lines
15 KiB
Markdown
# SOP — Character visual identity and appearance forms
|
||
|
||
Status: implemented. Covers how a character's avatar and concept (reference) sheet stay the same
|
||
person, and how one character holds several appearances ("forms") without splitting into several
|
||
characters or voices.
|
||
|
||
## Single source of truth
|
||
|
||
* Appearance facts come from the sheet's `physical`, `clothing`, `age_estimate`, `race_species`,
|
||
`gender`, `title`, `profession`, `archetype` (and the user overrides `visual_appearance`,
|
||
`visual_species`, `visual_costume`). Both prompts are built by
|
||
`_character_visual_facts` in `routes/conversation.py`; the active form is overlaid first
|
||
(`_character_form_sheet`, which never lets a form change its own `id`/`name`).
|
||
* `image_prompt` (portrait) and `concept_art_prompt` (reference sheet) are **deterministic** —
|
||
built server-side from those facts with fixed camera/lighting/period text. Sheet extraction no
|
||
longer asks the LLM to write `image_prompt` (it drifted from the concept sheet); the prompt is
|
||
generated right after extraction by `csAutoGenerateExternalPrompts`, and again before any
|
||
avatar-only or per-form generation. A hand-edited prompt (`*_user_edited`) is never overwritten.
|
||
* Only the two prompts describe an appearance. `voice_design_prompt` and the SillyTavern prompt
|
||
belong to the character as a whole, so they always live on the base sheet.
|
||
|
||
## Stale detection
|
||
|
||
`_character_identity_fingerprint` (server) and `_libIdentityFingerprint` (browser) hash the same
|
||
field list; a test runs the browser code in Node and compares, including list values and non-BMP
|
||
characters. The three `visual_*` overrides are appended only when set, so pairs made before they
|
||
existed keep a valid fingerprint. A pair needs review when its prompt version is not current, its
|
||
fingerprint is missing or differs, or `identity_consistent` is false. Regenerating **only** the
|
||
avatar or **only** the reference image sets `identity_consistent = false` on an existing pair,
|
||
because a lone regeneration produces a new face.
|
||
|
||
## Where the data lives
|
||
|
||
| | base appearance | a form |
|
||
|---|---|---|
|
||
| profile image | record `image` | `form.image` |
|
||
| reference image, version, fingerprint, digests, `identity_consistent` | `sheet.*` | `form.*` |
|
||
| Physical / Clothing text | `sheet.physical/clothing` | `form.physical/clothing` |
|
||
|
||
All visual results are written through one function (`_libSaveCharacterVisual`), which re-reads the
|
||
stored record, writes to the active form or the base, and saves. (It used to write the base
|
||
reference image to the record root, and the consistent-pair generator kept it in memory only, so a
|
||
base pair lost its reference image and metadata on reload.)
|
||
|
||
`GET /api/characters/visual-pair-review` checks the base pair and every form that has images, each
|
||
against its own metadata and its own fingerprint, and reports form pairs under the form name.
|
||
|
||
## Forms (max 8 per character, enforced in `core/validation.py`)
|
||
|
||
Add (starts as a copy of the base Physical/Clothing; only non-empty keys are stored because an
|
||
empty key would blank the base value when overlaid), rename (unique, ≤160 chars), delete (confirm;
|
||
the character, its voice and the base look stay), show, and generate matched images for one form or
|
||
for all forms. Generation rebuilds that form's prompts first (unless hand-edited), saves the pair on
|
||
that form, then restores whichever appearance was active. Editing Physical/Clothing while a form is
|
||
active changes only that form.
|
||
|
||
|
||
## Image prompt hygiene and the turnaround (1.34.x)
|
||
|
||
* **Setting.** Prompts describe the books as *The Dark Eye* high-fantasy medieval dress (about 900-1300 AD: wool and linen tunics, belted robes, leather, mail where the sheet names it). The earlier "early Renaissance 1420-1500" wording pulled uniforms, coats, hats and boots that read as musketeer/cowboy/wartime.
|
||
* **Wording.** Extracted costume text is rewritten at prompt time (`_period_wording`: Uniform, Weste, Hut, Sporen, Jacke, Schal … become surcoat, jerkin, hood, boots, tunic …). Stored sheets are not changed.
|
||
* **Injuries and dirt.** `_strip_injury_clauses` removes clauses about blood, wounds, bandages, burns, limping, torn clothes, soot and dirt from appearance, costume and the story moment; the rest of the sentence stays. The portrait baseline asks for clean skin and garments.
|
||
* **Variety.** Characters with armour or weapons get an individual garment cut and colour scheme (`_character_design_variation`, stable per name; colours the sheet names win) instead of one shared list.
|
||
* **The turnaround stays one T2I triptych.** Drawing front/side/rear one by one with the edit model was tried (2026-09-20) and reverted the next day: it returned three near-identical front views, mixed rendering styles between panels, cropped bodies, and copied the wireframe pose guide into the costume — exactly the failure the triptych was introduced to fix. The portrait/sheet face mismatch is still open and needs a different approach.
|
||
* **Props** get an object-only period sentence, so a tile no longer turns into a portrait of the character.
|
||
* Older saved prompts are cleaned the same way at render time. Existing image pairs are unchanged until regenerated.
|
||
|
||
|
||
## Automatic image check (`POST /api/character-image-check`)
|
||
|
||
A vision model reports **only what it sees** — figure count, the view of each figure left to right,
|
||
whether every head and foot is in frame, whether the figures are the same person, apparent gender,
|
||
ear shape (pointed/round/hidden), wounds, and background (plain/scenery). The expected values come from
|
||
the character sheet and the comparison happens in code (`core/character_image_check.py`). Asking a model
|
||
"is this a woman, as required?" invites agreement; asking "what gender is this?" does not, and the
|
||
expected gender is never revealed in the question.
|
||
|
||
* Kinds: `portrait`, `turnaround`, `scene`, `same_person` (profile vs. a sheet figure).
|
||
* **Fail-safe:** an unreachable checker, an unreadable answer, or a missing observation is a failure,
|
||
never a pass. An unreadable answer is returned verbatim (truncated) so it can be diagnosed.
|
||
* Thinking is switched off: a reasoning model spent the whole budget deliberating over a confusing image
|
||
and returned nothing.
|
||
* **Model:** the active LLM by default (`llm_model`), overridable with the `image_check_model` setting.
|
||
The model already loaded reads images, so a check never makes llama-swap unload it for a separate
|
||
vision model on a GPU with no room for both.
|
||
|
||
**Calibration (2026-09-21)** on seven panels whose defects were known (three front views, elf ears on a
|
||
human, a wireframe rendered as a figure, cut-off feet, a man where the sheet says woman, a forest behind a
|
||
studio portrait, and one correct same-person pair), run twice:
|
||
|
||
| model | correct | wrong | unreadable | per check |
|
||
|---|---|---|---|---|
|
||
| local Qwen3.8-27B (`default`) | 14/14 | 0 | 0 | ~4 s |
|
||
| Ling 3.0 Flash VL (`vision`, free, OpenRouter) | 8/14 | 0 | 6 | ~17 s |
|
||
|
||
Ling never judged wrongly but on every turnaround ignored the JSON-only instruction and answered in prose.
|
||
It also has free-tier rate limits and sends the images to a third party.
|
||
|
||
|
||
## Generation with checks and retries (face from the profile)
|
||
|
||
The generation job (`POST /api/character-generate-consistent-images`) now runs:
|
||
|
||
1. **Identity image** (text-to-image, full body). The profile picture is cut from it. **Portrait check**
|
||
on exactly that crop (one person, gender, ears, wounds, plain background). On failure: a new seed,
|
||
up to `image_check_max_attempts` (default 3), keeping the attempt with the fewest faults.
|
||
2. **Sheet** (turnaround triptych + story moment + props, one ComfyUI run). The profile's face is then
|
||
transferred with **ReActor** (`inswapper_128`, CodeFormer 0.5 restore): turnaround faces left to right
|
||
(front and side; the rear has none), and in the story panel only the *largest* face — the character in
|
||
the foreground, not an extra. **Checks:** turnaround, story panel, and same-person (profile vs. the front
|
||
view). Gender is judged *after* the face transfer, on what a viewer actually sees. Retried the same way.
|
||
3. The result carries `image_check` = `{verified, passed, attempts, failures[], checks{}}`, saved on the
|
||
sheet and shown on the character page: verified with no faults / faults still found (listed) / not
|
||
verified.
|
||
|
||
**If the checker itself is unavailable, retries stop at once** and the pair is marked *not verified*:
|
||
re-rolling images cannot fix a missing checker, and would burn up to three renders per character.
|
||
|
||
Why face transfer and not rendering the views from the portrait: the edit model keeps the source pose
|
||
(three front views, the pose guide copied into the costume — tried and reverted 2026-09-21). The
|
||
text-to-image triptych gets the poses right; swapping only the face afterwards fixes the identity.
|
||
|
||
Settings: `image_check_enabled` (default on), `image_check_max_attempts` (1–5, default 3),
|
||
`image_check_model` (default: the active LLM).
|
||
|
||
|
||
## The turnaround prompt is short and framing-first (2026-09-21)
|
||
|
||
Measured with turnaround-only renders scored by the image check (`scratchpad feet_experiment.py`):
|
||
|
||
| turnaround prompt | full body, head to shoes | plain backdrop |
|
||
|---|---|---|
|
||
| long compact prompt (~3 000 chars, framing rules mid-prompt), 1536x1024 | 0/5 | 2/5 |
|
||
| same, taller 1536x1280 canvas | 2/5 | 0/5 |
|
||
| same + "visible floor, contact shadow" wording | 0/5 | — |
|
||
| **short, framing first (~770 chars)**, 1536x1024 and 1536x1280 | **10/10** | **10/10** |
|
||
| the app's new prompt on Karyla, Darrag, Eolan (4 seeds each, after the fixes below) | **12/12** | **12/12** |
|
||
|
||
The 4-step Qwen-Image model follows the start of a prompt and loses the tail, so wording changes inside the
|
||
long prompt did nothing. The prompt now opens with the framing (three full-body views side by side,
|
||
head to soles, empty space above and below, plain white studio), then the character (subject capped at
|
||
700 chars), then only the rules a sheet needs, one short sentence each: leather-only armour, headwear,
|
||
elf ears (only for elves), child/teen proportions, clean skin and clean well-kept clothes.
|
||
|
||
What the validation runs found, each fixed and pinned by a test:
|
||
* **Passing states drawn as features** — "rote Stirn bei Wut, Tränen … vor Wut" became red marks (Darrag);
|
||
"Zunge streicht über die Lippen" a tongue sticking out (Eolan). Anger, tears, trembling, twitching,
|
||
blushing, sweating, licking are stripped like wounds.
|
||
* **Scars** — even a scar the sheet calls almost faded, with "fully healed, pale silvery line" added,
|
||
rendered as a fresh red gash, and that sentence put slashes on hands and robe. Scars are left out of
|
||
image prompts entirely; this model cannot draw a subtle one.
|
||
* **A dropped clause starting a sentence** left fragments ("und die rechte Augenbraue"): the whole sentence
|
||
now goes. "faded blue wool coat" stays (a costume, not a scar).
|
||
* Hedges "unklar" / "wird erwähnt" drop speculative sentences.
|
||
|
||
**Portrait (identity image) got the same treatment.** Old prompt ~2 800-3 700 characters, new ~875, framing
|
||
first ("full-length studio photograph … head to shoes … head in the upper third … plain white studio"). Identity
|
||
renders cropped exactly as the profile picture is, scored by the portrait check, 4 seeds each:
|
||
|
||
| | old prompt | new prompt |
|
||
|---|---|---|
|
||
| Karyla | 0/4 (scenery backdrop every time) | 4/4 |
|
||
| Darrag | 4/4 | 4/4 |
|
||
|
||
Both prompts share one rules line, which now again carries the per-character costume cut and colours for armed
|
||
characters (`_character_design_variation`; the first short turnaround had dropped it) and names the setting
|
||
("in the world of The Dark Eye"). The portrait asks for empty hands; weapon handling stays in the story prompt.
|
||
|
||
**The sheet follows the finished portrait.** After the portrait passes its check, the vision model describes what
|
||
it shows (`look`: hair colour/length/style, facial hair, eye colour, `look_phrase`, punctuation stripped and short),
|
||
and the turnaround and story prompts are rebuilt with "exactly as in the portrait: …". Found on Darrag, whose sheet
|
||
says nothing about hair: the portrait had shoulder-length hair and the turnaround a crop; the face transfer copied
|
||
the face but not the hair, and the same-person check failed on all three sheet attempts. Best effort: no
|
||
description, no change.
|
||
|
||
Open: robes can still come out with ragged hems; the story-moment prompt is still long.
|
||
|
||
|
||
## English prompt inputs (2026-10-03)
|
||
|
||
Sheets are extracted from German books, so their free text is mostly German. Image models do best with English,
|
||
and German leaked into keyword logic ("hat dünne Arme", "has thin arms", became a hood; "Mensch (oder Zwerg …)"
|
||
confused species checks). Every image-prompt builder therefore reads an **English copy** of the free text.
|
||
|
||
* **Layer:** `core/prompt_translation.py` (no network/DB code; the LLM is injected). Wiring:
|
||
`_english_prompt_inputs` in `routes/conversation.py`, called by `/api/character-generate-prompts` and
|
||
`/api/character-generate-consistent-images` *before* the builders and before species/age/dwarf/period logic run.
|
||
* **Fields:** species, appearance, clothing, age, title, profession, archetype, hair/eyes/build/marks, era,
|
||
mannerisms/capabilities, props/inventory, visual overrides, the chosen story-moment quote (as `visual_scene`)
|
||
and the book profile (genre, setting, era, visual context). Names and aliases are never translated.
|
||
* **Detection:** cheap stop-word/vocabulary heuristic; English or empty text is skipped without an LLM call.
|
||
* **Translation:** one batched JSON call per character through the app's existing LLM endpoint (`llm_url`,
|
||
model `prompt_translation_model` or `llm_model`; same retry/backoff helper), temperature 0.1, strict prompt
|
||
(faithful, keep names, no added or dropped traits, JSON only).
|
||
* **Validation (per field):** non-empty, not an error/refusal, length sanity, not an unchanged German echo,
|
||
not still German, names present in the output when present in the source, list shape kept. A rejected field
|
||
keeps its original text; the other fields still use their translation.
|
||
* **Cache:** table `prompt_translation_cache` in the existing SQLite DB (derived data, separate from
|
||
`characters`), key = `sha256(source)|lang|model|prompt version`. Editing the source text, switching the
|
||
model or bumping `PROMPT_VERSION` misses automatically. Records are never rewritten; the UI keeps showing
|
||
the German source of truth. The identity fingerprint and the image checks keep using the original sheet.
|
||
* **Fallback:** LLM down, invalid output or flag off: the original text is used (the German keyword rules
|
||
stay as the fallback), a warning is logged, and the response carries `prompt_translation`
|
||
(`used_untranslated_text`, `translated_fields`, `untranslated_fields`, `cache_hits`, `error`). Image
|
||
generation is never blocked.
|
||
* **Flag:** env `PROMPT_TRANSLATION=off` (or settings key `prompt_translation: false`); default on.
|
||
* **Dry run:** `POST /api/character-prompt-preview` with `{id}` or `{name, book, sheet, book_profile}` returns the
|
||
prompts built from the original and from the English text plus the changed fields; it generates nothing and
|
||
writes no record (only the translation cache).
|
||
* **Side fixes:** `_character_age_category` now reads `age_estimate` (sheets store that key; a boy used to render
|
||
adult); `_period_wording` no longer turns German "hat" (has) into a hood.
|