77 KiB
User manual
Living manual. Every feature updates this file in the same commit as the code.
Sections for the mature features (voice library, Read Aloud, audiobook casting, Rehearser) are still to be written — this file was created alongside the magazine feature and currently documents that area only.
Design a Voice: attribute chips
Purpose: build a voice description quickly. Use: open Design a Voice, press chips (gender, age, pitch, pace, accent/region, tone/timbre, emotion). Each chip adds its phrase to the description; pressing it again removes only that phrase. Gender, age, pitch and pace allow one choice each. Free text: type anything in the box; chips never overwrite your text and are not a limit on what you can describe. Keyboard/touch: chips are buttons (Tab, Space/Enter, aria-pressed, 44 px targets). Languages: chip labels follow the UI language (German/English); the inserted phrase is English. Offline/privacy: purely local, nothing is sent until you generate.
Voice export and import bundles
Purpose: back up a voice or move it between machines (DGX and NUC) as one .voice.zip file containing the reference audio, transcript, settings (seed, design prompt, rating, tags, source & consent, audio problems), and optionally the picture and the consent recording.
Prerequisites: Voice library open. Export needs at least one checked voice.
Steps: tick the checkboxes of the voices, press "Export selected"; one <voice>.voice.zip is downloaded per voice. On the other machine press "Import bundle" and choose one or more .voice.zip files.
Expected result: the voice appears in the library. If the name is already taken the import gets a suffix (_2, _3); an identical voice that is already there is reported as "already present". Existing voices are never overwritten.
Errors: damaged, tampered, oversized or unexpected files are rejected with a message (for example "Hash mismatch"); nothing is imported then. Messages appear as toasts, which screen readers announce.
Mobile / accessibility: both buttons are at least 44 px high and keyboard operable; the file chooser is the browser's own.
Offline: works fully offline. Privacy: a bundle contains the voice, its source/consent notes and any consent recording; share it only with people you trust.
Voice source & consent
Purpose: record where a voice comes from and why you may use it (own voice, speaker consented, licensed, public domain, unknown).
Prerequisites: a voice in the Voice library. For a spoken consent statement: a microphone and browser permission.
Steps: open a voice in the Voice library, scroll to "Source & consent", pick the basis, add source (URL or description, never passwords), licence and a note, press "Save source & consent". Optionally press "Record spoken consent", say who you are and what you consent to, press "Stop recording"; the clip is saved and can be replayed.
Result: the "Source unknown" warning badge disappears from the list once a basis other than "Unknown" is saved. Imported voices get the source URL prefilled but stay "Unknown" until you decide.
Errors: invalid values (bad basis, over-long text, non-http(s) URL or URL with credentials) are reported next to the Save button. If the microphone is denied or missing, type a written consent statement into the note field instead.
Mobile/accessibility: all controls have labels and at least 44 px targets; status messages are announced; the typed note is a full alternative to recording.
Offline: works offline (local server only).
Privacy: everything stays on this server in the voice's .meta.json and <voice>.consent file; nothing is sent to external services. Generation is not blocked by this field (yet).
Voice health: audio problems
Purpose: reference audio is checked for silence (over 1.5 s inside the clip), clipping (peaks at full scale for 0.1 s or more) and level jump (15 dB or more between adjacent half-second windows).
Where: (1) Settings, Routing, Test: the result line shows "Audio problems: clipping at 2s-2.4s (peak at 0.0 dBFS); ..." (first 8). (2) Try-out, after selecting a library voice: the voice panel shows the same text under the warnings (screen-reader announced; wording carries the meaning, not colour). Each kind also adds a warning.
When computed: on import (stored in the voice's meta as audio_problems), and fresh on demand via GET /api/voices/{id}/health (cached by file modification time). The voice list only returns the stored result.
Errors: unreadable audio yields no problems; only the first 120 s are scanned. Privacy/offline: local analysis only.
Seed Finder: Auto-pick best
Purpose. Finds a good seed for a voice without listening to every sample first.
Prerequisites. A voice with a reference clip (Voice Clone backend); a speech-to-text engine is optional.
Steps. Open the voice, then Seed Finder. Set the test sentence and the seed range (at most 16 seeds are tried), then click Auto-pick best. The server generates each candidate one after another, scores it and the best row is marked "Best". Play any row, then click Use seed N to save it. Nothing is saved automatically.
Score (0-100). Weighted mean of: spoken-back word accuracy (50%, text read back by speech-to-text), loudness and clipping (20%), audio problems such as long silences and sudden level jumps (30%). Weights and penalties are constants in core/seed_score.py.
Errors. If speech-to-text is down, the result says so and scores by loudness and audio problems only. A failed seed shows its error and is skipped. Cancel stops after the running candidate; the voice configuration is always restored.
Mobile and accessibility. Progress and the result are announced in a polite live region; the button is keyboard operable. German and English labels.
Offline. Needs the TTS server (and optionally the STT server) on your network. Privacy. The sentence is sent only to your own TTS/STT servers.
Audiobook casting (Read Aloud → Cast as Audiobook)
Purpose
Turn a book into a script with one narrator and one voice per character, ready for the Rehearser and MP3 export.
Prerequisites
- A book open in Read Aloud (PDF or text) and an LLM configured in Settings.
Step by step
- Open the book, then Cast as Audiobook.
- The app finds the quotation marks in the text and splits it into narration and spoken lines by itself — this part never changes a word of the book. The LLM is then asked only who speaks each spoken line, in passages of a few thousand characters. A whole novel takes roughly ten minutes (the previous method took hours).
- Progress shows per passage; Stop Casting keeps what is done and Continue casting resumes.
- Review the cast as before. Recast unknown re-reads only the lines still marked Unknown, with more context; a line the model cannot place stays Unknown rather than getting a guessed voice. Verify (the second quality run) reads every spoken line a second time with different passage boundaries — about as long as the first cast — and never changes a speaker: lines both readings agree on are marked confirmed (about 4 in 5 of those were right in testing), and lines where they disagree keep the first reading, are flagged in the review list and show what the other reading said. The percentage next to each line means what it says in testing: confirmed lines were about 82 % right, disputed lines about 48 %, lines not yet verified about 76 %. Use the flags to decide where to spend your review time. Neither step can change the book's text, and a stopped or failed run leaves the cast as it was.
Expected result: every word of the book appears exactly once, quoted speech is typed as dialogue and everything else as narration. When the text only refers to a person by role ("der Fremde", "die Baronin"), the line is labelled with that role; later passes merge it into the named character when the book itself ties the two together (e.g. "der Inquisitor Marcian"). Some role labels stay separate — use the character merge in the cast view for those.
Errors, limits and privacy
- If the LLM cannot be reached for a passage, its lines are named from the speech tags around them where possible and otherwise marked Unknown; nothing is lost.
- A book whose speech is not marked with quotation marks (»…«, „…“, “…”, «…») uses
the previous whole-passage method automatically. To force the previous method in
this browser:
localStorage['ab-span-mode'] = '0'. - The custom attribution prompt in Settings only applies to the older whole-passage method; span mode uses its own fixed instructions (it only asks who speaks each line).
- Book text goes to the LLM you configured, exactly as before; nothing else is sent anywhere. Nothing works offline without a reachable LLM.
- Accuracy: measured against hand-corrected casts of three novels, roughly 90–92 % of all words end up with the right voice and about three quarters of spoken words with the right speaker. Similar-looking characters in the same scene are the usual mix-up; check those in review.
Character concept sheet (Konzeptbild)
Purpose: one picture per character for costume continuity: the character from the front, the side and the back on top; below, the same character in a moment from the book, and their own items (up to three, e.g. Marcian's crossbow).
Prerequisites: a local ComfyUI (Settings > Engines > Image Generation) and the character's Image and Concept Art prompts (Cast Studio > character > Generation Prompts).
Step by step: in Cast Studio tick the characters and choose Consistent profile + reference, or open a character and regenerate its Konzeptbild. Pick Cinematic or Concept art in the style menu first.
Expected result: the lower pictures show the same person as the poses (face, hair, armour or clothing, colours), in the medieval high-fantasy world of the book (stone, timber, tents, torch or daylight, other people in medieval dress), and the item lies on a plain neutral background as an isolated prop, with no hands, feet or other body parts, made in the style and colours of its owner's costume. They contain no writing, captions or modern things. The face in the story moment is taken from the profile picture.
Errors and limits: the fast local model still gets details wrong now and then (a crossbow with a misshapen stock, a second weapon in the background, Concept art coming out close to photographic). Regenerate the sheet; each run uses a new seed. The automatic image check looks at gender, ears and wounds in the story moment, not at modern objects or text, so look at the lower pictures yourself. Sheets made before 2026-10-02 keep their old lower pictures until they are regenerated.
Species: the body shape (for example short dwarf proportions with a full beard) follows only the species the character's own Race / Species field states first. A note in brackets such as "Mensch (oder Zwerg, je nach Kontext …)" or a long remark about other characters is ignored, so such a character is drawn as a human. If a character really is a dwarf but the field says "Mensch (…)", change the field to "Zwerg" and regenerate.
Story moment: the lower picture uses a quote from the character's Sources that names this character (by name, real name or a real alias such as "Murlok von Mengbilla"; "der Zwerg" alone does not count). A quote that only names someone else is never used; without a fitting quote the character's own mannerisms or abilities are shown instead. To change that fallback, edit the character's Mannerisms.
Age: a child or teenager is drawn as one. The app reads the age estimate ("12 Jahre", "zwölfjährig", "fast acht Sommer", "15 years old"), age quotes in Sources, and words like "Kind", "Junge", "Mädchen", "boy", "girl". "Junger Mann" / "Junge Frau" stay adults. If a hedged estimate ("Junges Mädchen / Junger Mann") is drawn wrongly, write the age as a number in Estimated age and regenerate.
Privacy and offline: only the prompts and the profile picture go to your own ComfyUI; nothing leaves your network. Without a reachable ComfyUI nothing is generated.
Accessibility and mobile: the sheet is shown as one image with the character's name as its alternative text; the character's written profile next to it carries the same information.
English prompts for character images
Purpose: the image models work best with English, but character sheets from German books are German. Before a profile or reference-sheet prompt is built, the app translates the character's text (species, appearance, clothing, age, items, story moment, the book's setting) to English with your own LLM.
Prerequisites: the LLM from Settings. Without it the original German text is used.
Usage: nothing to do. Generate prompts or images as before. Your character sheet keeps its German text
and is never changed; translations are cached, so a character is translated once and again only after its
text changes. To inspect the result without generating anything, call POST /api/character-prompt-preview
with a character id.
Expected result: English prompts; the response and the finished image job contain prompt_translation.
Errors: if the LLM is unreachable or its answer fails validation, the original text is used for the
affected fields, a warning is logged, prompt_translation.used_untranslated_text is true, and generation
continues. Set PROMPT_TRANSLATION=off to disable translation entirely.
Mobile, accessibility, offline: no interface change. Offline (no LLM) falls back to German text.
Privacy: character text goes only to the LLM you configured; cached translations stay in the local database.
Distinct faces for each character
Purpose: characters of one book used to look like members of one family, because most sheets describe no face and the image model then draws its default one. Every character now gets its own face description, so the cast is easy to tell apart. What the book says about a character always wins.
Prerequisites: none. It is part of generating character prompts and images.
Usage: generate profile/reference images as before. The prompt now contains a face: clause such as
face: oval face, narrow pointed chin, straight nose, wide-set grey eyes, ..., chestnut hair, shoulder-length wavy hair.
Parts the sheet states (for example "schwarzhaarig, blaue Augen") are kept; the other parts are chosen by the app,
fitting species, age and gender (no beards for women or children, no elf traits on humans, grey hair for the old),
and spread across the characters of the book. The choice is stable: the same character always gets the same face,
and adding new characters does not change existing ones. Relatives named in the relationships field (brother,
father ...) may share eye colour, hair colour, nose or jaw.
Override: to fix a face, write it in the sheet field visual_face (free text, English works best), for example
narrow face, hooked nose, one pale grey eye, black hair tied back. It replaces the generated face. Describing
features in the physical appearance also works; those words are kept.
Preview without generating: POST /api/character-face-preview with {"book": "<book title>"} returns every
character's face (and where each part came from) plus a similarity matrix and a before/after score;
POST /api/character-prompt-preview shows the finished prompt including face_clause. Nothing is generated or saved.
Expected result: clearly different faces within a book. Existing images stay as they are until you regenerate them.
Errors: if the cast cannot be read, the character gets a face from its own sheet alone. Non-human sheets (a raven, the narrator) get no face clause.
Mobile, accessibility, offline: no interface change; works offline (no LLM needed for the face itself).
Privacy: nothing leaves the machine; profiles are derived from the stored sheets and not saved.
Keeping characters apart (profiles and the character library)
Purpose: each character's profile holds only that character's facts.
What the app does: two profiles are treated as the same person only when they share a real name. Labels many people share ("Zwerg", "der Zwerg", "Mann", "Fremder", "Soldat", "Ork-General") never join profiles. A profile is saved to the record with exactly its name. When the cast is known, another cast member's name is never kept as an alias or first/last name (relations such as "Tochter des Promos" stay), and notes in which the model talks to itself ("Warte: ...", "Der Text sagt ...") are removed.
Checking an existing library (administrators): on the server, run
python3 scripts/audit_character_records.py for a read-only report of copied profiles,
names of other characters in aliases, leaked notes and foreign quotes. To see what a
cleanup would change, run python3 scripts/clean_character_records.py --out <folder>:
it writes report.md and changes nothing. Only with --apply (after you have read the
report) does it save a backup to the folder and change the listed records; copied
profiles are only reported, you decide which one to keep (delete or recast the other).
Errors: a profile that was copied or mixed up before this fix keeps its wrong text until you edit it, delete the copy, or recast that character.
Mobile, accessibility, offline: no interface change. Privacy: the scripts run locally against your own database; nothing is sent anywhere.
Characters: all books or one book
Purpose: two views of the same character library, so you always know which book you are working in.
- All characters (all books) (sidebar Library > Characters / Cast, or the Library Cast tab): every character of every book, one band per book with a heading and a count. The first book is open, the others are closed and only drawn when opened. Search all books (name, alias, archetype, book) opens every band with matches and shows "2 of 9". Cards/Table, Sort and every per-book action stay in each band header.
- Studio > Voices (step 3): after you pick a book in Source, only that book's characters are shown, under a Book: