Initial: LLM Wiki stack

2026-06-16 16:04:26 +00:00 · 2026-06-16 16:04:26 +00:00 · 9745214941
commit 9745214941
13 changed files with 542 additions and 0 deletions
--- a/.env.example
+++ b/.env.example
@ -0,0 +1,9 @@
+# LLM Wiki — Environment Variables
+# Copy to .env and customize for your deployment.
+
+# Port for the wiki viewer (default: 8787)
+WIKI_PORT=8787
+
+# Wiki metadata
+WIKI_TITLE=My LLM Wiki
+WIKI_DESCRIPTION=A personal knowledge base built with LLMs
--- a/12
+++ b/12
@ -0,0 +1,12 @@
+FROM squidfunk/mkdocs-material:latest
+LABEL app="llm-wiki" description="Karpathy's LLM Wiki — persistent knowledge base maintained by LLM agents"
+
+# Copy the entire wiki project into /docs (mkdocs WORKDIR)
+COPY mkdocs.yml /docs/mkdocs.yml
+COPY docs/ /docs/docs/
+COPY schema/ /docs/schema/
+
+# The base image already has:
+#   WORKDIR /docs
+#   ENTRYPOINT ["/sbin/tini", "--", "mkdocs"]
+#   CMD ["serve", "--dev-addr=0.0.0.0:8000"]
--- a/README.md
+++ b/README.md
@ -0,0 +1,71 @@
+# LLM Wiki by Karpathy — Docker / Portainer Deployment
+
+A Docker-based deployment of [Andrej Karpathy's LLM Wiki pattern](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f).
+
+> **A persistent, compounding knowledge base maintained by LLM agents.**
+> You curate sources and ask questions — the LLM does all the bookkeeping.
+
+## Architecture
+
+```
+llm-wiki/
+├── docker-compose.yml       # Portainer-compatible stack
+├── mkdocs.yml               # MkDocs configuration (nav, theme, plugins)
+├── .env.example             # Environment variables
+├── docs/docs/
+│   ├── index.md             # Home page
+│   ├── wiki/                # LLM-maintained content pages
+│   │   ├── index.md         # Wiki index
+│   │   ├── entities/        # Entity pages (people, places, systems)
+│   │   ├── concepts/        # Concept pages
+│   │   ├── comparisons/     # Comparison pages
+│   │   └── syntheses/       # Integrated overviews
+│   ├── sources/             # Ingested source summaries
+│   ├── raw/                 # Raw source files (immutable)
+│   ├── schema/              # AGENTS.md (schema for the LLM agent)
+│   └── log.md               # Append-only changelog
+└── schema/
+    └── AGENTS.md            # Master schema file (paste to your LLM agent)
+```
+
+## Deployment
+
+### Portainer (recommended)
+
+1. Copy the `docker-compose.yml` content
+2. Portainer → Stacks → Add Stack → **Web Editor**
+3. Paste the compose content
+4. Stack name: `llm-wiki`
+5. Environment variables (optional):
+   - `WIKI_PORT=8787`
+6. Click **Deploy the stack**
+
+### Docker Compose CLI
+
+```bash
+cp .env.example .env
+docker compose up -d
+```
+
+Then visit **http://<host>:8787**
+
+## How to Use
+
+### 1. Add content
+Drop markdown files into `docs/docs/raw/` — articles, notes, papers, etc.
+
+### 2. Tell your LLM agent
+Give your LLM agent (Hermes, Claude Code, Codex) the AGENTS.md schema:
+
+> "Here is my LLM Wiki schema: [paste schema/AGENTS.md]. Please read raw/ and ingest the new sources."
+
+### 3. Explore
+Browse the wiki at http://<host>:8787 — search, follow links, see the structure grow.
+
+### 4. Lint
+Periodically ask your agent to health-check the wiki for contradictions, orphans, and gaps.
+
+## Credits
+
+- **[Andrej Karpathy](https://karpathy.ai/)** — original LLM Wiki concept
+- **[MkDocs Material](https://squidfunk.github.io/mkdocs-material/)** — wiki viewer theme
--- a/docker-compose.yml
+++ b/docker-compose.yml
@ -0,0 +1,40 @@
+# LLM Wiki — Docker Stack
+# Karpathy's pattern for LLM-maintained personal knowledge bases
+# https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f
+#
+# Architecture (three layers, see schema/AGENTS.md):
+#   raw/      — your curated source documents (immutable)
+#   wiki/     — LLM-generated interlinked markdown pages
+#   schema/   — AGENTS.md tells your LLM how to maintain the wiki
+#
+# The mkdocs-material image runs `mkdocs serve` from /docs,
+# serving the wiki as a beautiful, searchable website.
+#
+# Portainer deployment: Web Editor → paste this docker-compose.yml → Deploy.
+#
+# ⚠️ NOTE: bind mounts use absolute paths (/opt/data/workspace/...)
+#    because this is an NFS-shared volume between PVE host and NUC.
+#    Adjust paths if your NUC has a different mount point.
+
+services:
+  wiki:
+    image: squidfunk/mkdocs-material:latest
+    container_name: llm-wiki
+    restart: unless-stopped
+    ports:
+      - "${WIKI_PORT:-8787}:8000"       # Host port → mkdocs dev server
+    volumes:
+      # Mount the entire project root → /docs (single clean mount)
+      - /opt/data/workspace/docker/llm-wiki:/docs
+    environment:
+      - MKDOCS_VERBOSE=true
+      - MKDOCS_LIVERELOAD=true
+    healthcheck:
+      test: ["CMD", "wget", "-qO-", "http://localhost:8000"]
+      interval: 30s
+      timeout: 10s
+      retries: 3
+      start_period: 10s
+    labels:
+      - "app=llm-wiki"
+      - "description=Karpathy's LLM Wiki — persistent knowledge base"
--- a/docs/index.md
+++ b/docs/index.md
@ -0,0 +1,51 @@
+# Welcome to Your LLM Wiki 🧠
+
+> A **persistent, compounding knowledge base** maintained by LLM agents.
+>
+> *"The wiki is a persistent, compounding artifact. The cross-references are already there. The contradictions have already been flagged. The synthesis already reflects everything you've read."* — [Andrej Karpathy](https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f)
+
+---
+
+## Quick Start
+
+### 1. Add a source
+Drop an article, paper, or note into **`raw/`**, then tell your LLM agent:
+
+> "I added a new source to raw/. Please ingest it into the wiki."
+
+### 2. Ask a question
+Your LLM will search the wiki and synthesize an answer with citations.
+
+> "What do we know about [topic]?"
+
+### 3. Save insights back
+Good answers can be filed into the wiki:
+
+> "Write that comparison as a new wiki page and update the index."
+
+### 4. Lint periodically
+> "Health-check the wiki. Look for contradictions, stale claims, orphan pages, and missing cross-references."
+
+---
+
+## Architecture
+
+| Layer | Directory | Purpose |
+|-------|-----------|---------|
+| **Raw Sources** | `raw/` | Immutable source documents (articles, papers, notes) |
+| **The Wiki** | `wiki/` | LLM-generated markdown pages — entities, concepts, comparisons |
+| **The Schema** | `schema/AGENTS.md` | Configuration that tells the LLM how to maintain the wiki |
+
+### Special files
+- **`index.md`** — content catalog (LLM-maintained)
+- **`log.md`** — chronological changelog (LLM-maintained)
+- **`tags.md`** — auto-generated tag index (for `mkdocs-material`)
+
+---
+
+## Tips
+
+- Use **Obsidian Web Clipper** to save articles as markdown into `raw/`
+- Run `docker compose --profile setup run init` to initialize the structure
+- The wiki viewer runs at **`http://<host>:8787`**
+- Your LLM agent (Hermes, Codex, Claude Code) does all the maintenance — you just curate and explore
--- a/docs/log.md
+++ b/docs/log.md
@ -0,0 +1,10 @@
+# Changelog
+
+> Chronological record of ingests, queries, and maintenance.
+> Each entry starts with a consistent date prefix for easy parsing with Unix tools.
+
+## [2026-06-16] init | Wiki created
+
+- Initial wiki structure created via docker-compose deployment
+- mkdocs-material wiki viewer deployed on port 8787
+- Schema file (AGENTS.md) written
--- a/docs/raw/index.md
+++ b/docs/raw/index.md
@ -0,0 +1,9 @@
+# Raw Sources
+
+> Your curated collection of source documents — articles, papers, notes, images, data files.
+>
+> These are **immutable**: the LLM reads from them but never modifies them. This is your source of truth.
+>
+> To add a source, place it in this directory and tell the LLM agent to ingest it.
+
+[Back to Home](../index.md)
--- a/docs/schema/AGENTS.md
+++ b/docs/schema/AGENTS.md
@ -0,0 +1,111 @@
+# AGENTS.md — LLM Wiki Schema
+
+> This file tells the LLM agent how to maintain the wiki.
+> Share this with your LLM agent (Hermes, Codex, Claude Code, etc.) so it knows the conventions, structure, and workflows.
+
+## Wiki Structure
+
+```
+docs/
+├── index.md              # Welcome page + quick-start guide
+├── mkdocs.yml            # MkDocs configuration (theme, nav, plugins)
+├── wiki/                 # LLM-generated content pages
+│   ├── index.md          # Wiki index / category listing
+│   ├── entities/         # Entity pages (people, places, systems)
+│   ├── concepts/         # Concept pages (ideas, theories, patterns)
+│   ├── comparisons/      # Comparison / analysis pages
+│   └── syntheses/        # Integrated overviews across sources
+├── sources/              # Ingested source summaries
+│   └── index.md          # Source catalog (table)
+├── raw/                  # Raw source files (immutable)
+│   └── index.md          # Raw directory index
+└── log.md                # Append-only changelog
+```
+
+## Conventions
+
+### Page format
+```markdown
+---
+tags: [tag1, tag2]
+created: YYYY-MM-DD
+updated: YYYY-MM-DD
+sources: [source-filename-1, source-filename-2]
+---
+
+# Page Title
+
+Summary paragraph.
+
+## Section
+
+Content with [[wiki/entities/related-page]] internal links.
+
+## Sources
+- [Source Title](../sources/source-summary.md)
+```
+
+### Internal links
+Use relative markdown links: `[[wiki/concepts/some-topic]]` or `[text](../wiki/entities/page.md)`.
+Always link to wiki pages, never to raw sources, from within wiki content.
+
+### Tags
+Use lowercase, hyphenated tags in frontmatter. Add or update tags as the wiki grows.
+
+### Dates
+Use `YYYY-MM-DD` format consistently.
+
+## Workflows
+
+### Ingest a new source
+1. Read the source file from `raw/`
+2. If images are referenced, also examine them with the vision tool for additional context
+3. Discuss key takeaways with the user
+4. Write a summary page in `sources/` (with link to the raw file)
+5. Create or update relevant entity/concept pages in `wiki/`
+6. Update `wiki/index.md` and `sources/index.md`
+7. Append an entry to `log.md` with the `## [YYYY-MM-DD] ingest | Title` format
+8. Update `mkdocs.yml` `nav:` section if new top-level pages were added
+
+### Answer a query
+1. Read `sources/index.md` and/or `wiki/index.md` to find relevant pages
+2. Read the relevant pages
+3. Synthesize an answer with citations to wiki pages
+4. Offer to save the answer as a new wiki page: "Shall I file this as a comparison/synthesis page?"
+
+### Lint the wiki
+Periodically check for:
+- Contradictions between pages
+- Stale claims that newer sources have superseded
+- Orphan pages with no inbound links
+- Important concepts mentioned but lacking their own page
+- Missing cross-references
+- Data gaps that could be filled with a web search
+
+### Write a page
+Pages should be informative, well-structured, and cross-linked.
+Use admonitions for notes, warnings, or open questions:
+> [!note] Open question
+> The relationship between X and Y is not yet clear from available sources.
+
+> [!warning] Conflicting claims
+> Source A claims Z, but source B says ¬Z. This needs further investigation.
+
+## Tools available
+
+- `raw/` directory: read-only source files
+- `wiki/` directory: create, update, or reorganize wiki pages
+- `sources/` directory: write source summaries
+- `log.md`: append entries (never edit existing entries)
+- `mkdocs.yml`: update nav section as the wiki grows
+- Web search: fill data gaps identified during lint
+- Terminal: git operations for version control
+
+## Git workflow
+
+The wiki is a git repository. After each ingest or significant update:
+```bash
+cd /docs && git add -A && git commit -m "ingest: [title] — YYYY-MM-DD"
+```
+
+*This schema was initialized from Karpathy's LLM Wiki pattern: https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f*
--- a/docs/sources/index.md
+++ b/docs/sources/index.md
@ -0,0 +1,14 @@
+# Sources
+
+> Your curated collection of source documents. The LLM reads from these but never modifies them — this is your source of truth.
+>
+> Each ingested source gets a summary page here with links to the raw file and the wiki pages it informed.
+
+## Ingested Sources
+
+| # | Source | Date | Type | Category |
+|---|--------|------|------|----------|
+
+---
+
+*This index is maintained by the LLM agent.*
--- a/docs/tags.md
+++ b/docs/tags.md
@ -0,0 +1,8 @@
+# Tags
+
+> Auto-generated index of tags used in the wiki.
+> Maintained by the `tags` plugin in mkdocs.yml.
+
+---
+
+<!-- material/tags --> 
--- a/docs/wiki/index.md
+++ b/docs/wiki/index.md
@ -0,0 +1,16 @@
+# Wiki
+
+> LLM-generated entity pages, concept pages, comparisons, and syntheses.
+>
+> The LLM owns this layer entirely. It creates pages, updates them when new sources arrive, maintains cross-references, and keeps everything consistent.
+
+## Categories
+
+- **Entities** — people, places, organizations, systems
+- **Concepts** — ideas, theories, patterns, frameworks
+- **Comparisons** — side-by-side analyses of related topics
+- **Syntheses** — integrated overviews drawing from multiple sources
+
+---
+
+*This index is maintained by the LLM agent.*
--- a/mkdocs.yml
+++ b/mkdocs.yml
@ -0,0 +1,80 @@
+# LLM Wiki — MkDocs Configuration
+# Serves the wiki as a beautiful documentation site with full-text search.
+# Edit this to customize the theme, nav structure, or plugins.
+
+site_name: "LLM Wiki"
+site_description: "Personal knowledge base — maintained by LLM agents"
+site_url: ""
+repo_url: ""
+edit_uri: ""
+docs_dir: docs
+site_dir: site
+
+theme:
+  name: material
+  language: en
+  palette:
+    primary: indigo
+    accent: teal
+  features:
+    - navigation.tabs
+    - navigation.sections
+    - navigation.expand
+    - navigation.top
+    - navigation.indexes
+    - search.highlight
+    - search.suggest
+    - content.code.copy
+    - content.tooltips
+
+plugins:
+  - search:
+      lang: en
+  - tags:
+      tags_file: tags.md
+
+markdown_extensions:
+  - abbr
+  - admonition
+  - attr_list
+  - def_list
+  - footnotes
+  - md_in_html
+  - toc:
+      permalink: true
+  - pymdownx.highlight:
+      anchor_linenums: true
+  - pymdownx.inlinehilite
+  - pymdownx.snippets
+  - pymdownx.superfences
+  - pymdownx.tabbed:
+      alternate_style: true
+  - pymdownx.details
+  - pymdownx.tasklist:
+      custom_checkbox: true
+  - pymdownx.critic
+  - pymdownx.caret
+  - pymdownx.keys
+  - pymdownx.mark
+  - pymdownx.tilde
+
+extra:
+  social: []
+
+# === Wiki Navigation ===
+# The LLM agent updates this section as the wiki grows.
+# Categories mirror Karpathy's LLM Wiki architecture:
+#   - Overview  → index, synthesis, tags
+#   - Wiki      → entity pages, concept pages, comparisons
+#   - Sources   → ingested documents (one page per source)
+#   - Raw       → direct access to raw source content (linked, not separate)
+
+nav:
+  - Home: index.md
+  - Wiki:
+      - wiki/index.md
+  - Sources:
+      - sources/index.md
+  - Raw Sources: raw/
+  - Schema: schema/AGENTS.md
+  - Log: log.md
--- a/schema/AGENTS.md
+++ b/schema/AGENTS.md
@ -0,0 +1,111 @@
+# AGENTS.md — LLM Wiki Schema
+
+> This file tells the LLM agent how to maintain the wiki.
+> Share this with your LLM agent (Hermes, Codex, Claude Code, etc.) so it knows the conventions, structure, and workflows.
+
+## Wiki Structure
+
+```
+docs/
+├── index.md              # Welcome page + quick-start guide
+├── mkdocs.yml            # MkDocs configuration (theme, nav, plugins)
+├── wiki/                 # LLM-generated content pages
+│   ├── index.md          # Wiki index / category listing
+│   ├── entities/         # Entity pages (people, places, systems)
+│   ├── concepts/         # Concept pages (ideas, theories, patterns)
+│   ├── comparisons/      # Comparison / analysis pages
+│   └── syntheses/        # Integrated overviews across sources
+├── sources/              # Ingested source summaries
+│   └── index.md          # Source catalog (table)
+├── raw/                  # Raw source files (immutable)
+│   └── index.md          # Raw directory index
+└── log.md                # Append-only changelog
+```
+
+## Conventions
+
+### Page format
+```markdown
+---
+tags: [tag1, tag2]
+created: YYYY-MM-DD
+updated: YYYY-MM-DD
+sources: [source-filename-1, source-filename-2]
+---
+
+# Page Title
+
+Summary paragraph.
+
+## Section
+
+Content with [[wiki/entities/related-page]] internal links.
+
+## Sources
+- [Source Title](../sources/source-summary.md)
+```
+
+### Internal links
+Use relative markdown links: `[[wiki/concepts/some-topic]]` or `[text](../wiki/entities/page.md)`.
+Always link to wiki pages, never to raw sources, from within wiki content.
+
+### Tags
+Use lowercase, hyphenated tags in frontmatter. Add or update tags as the wiki grows.
+
+### Dates
+Use `YYYY-MM-DD` format consistently.
+
+## Workflows
+
+### Ingest a new source
+1. Read the source file from `raw/`
+2. If images are referenced, also examine them with the vision tool for additional context
+3. Discuss key takeaways with the user
+4. Write a summary page in `sources/` (with link to the raw file)
+5. Create or update relevant entity/concept pages in `wiki/`
+6. Update `wiki/index.md` and `sources/index.md`
+7. Append an entry to `log.md` with the `## [YYYY-MM-DD] ingest | Title` format
+8. Update `mkdocs.yml` `nav:` section if new top-level pages were added
+
+### Answer a query
+1. Read `sources/index.md` and/or `wiki/index.md` to find relevant pages
+2. Read the relevant pages
+3. Synthesize an answer with citations to wiki pages
+4. Offer to save the answer as a new wiki page: "Shall I file this as a comparison/synthesis page?"
+
+### Lint the wiki
+Periodically check for:
+- Contradictions between pages
+- Stale claims that newer sources have superseded
+- Orphan pages with no inbound links
+- Important concepts mentioned but lacking their own page
+- Missing cross-references
+- Data gaps that could be filled with a web search
+
+### Write a page
+Pages should be informative, well-structured, and cross-linked.
+Use admonitions for notes, warnings, or open questions:
+> [!note] Open question
+> The relationship between X and Y is not yet clear from available sources.
+
+> [!warning] Conflicting claims
+> Source A claims Z, but source B says ¬Z. This needs further investigation.
+
+## Tools available
+
+- `raw/` directory: read-only source files
+- `wiki/` directory: create, update, or reorganize wiki pages
+- `sources/` directory: write source summaries
+- `log.md`: append entries (never edit existing entries)
+- `mkdocs.yml`: update nav section as the wiki grows
+- Web search: fill data gaps identified during lint
+- Terminal: git operations for version control
+
+## Git workflow
+
+The wiki is a git repository. After each ingest or significant update:
+```bash
+cd /docs && git add -A && git commit -m "ingest: [title] — YYYY-MM-DD"
+```
+
+*This schema was initialized from Karpathy's LLM Wiki pattern: https://gist.github.com/karpathy/442a6bf555914893e9891c11519de94f*