update: 24B/8B Modell-Tests, :wiki-Befehl, wiki-Doku, Modell-Vergleich
- MODELS.md: 8B vs 24B Vergleich, :wiki-Doku, Entscheidungsbaum
- README.md: 24B mit ⚠️ Manuell markiert (FILE:-Pattern unzuverlässig)
- SESSION_HANDOVER.md: Neue Session 14:30-18:00
- TEST_SCENARIO.md: Referenz-Testszenario
- crowdcode.js: 24B-Kompatibilität (flexible FILE:-Erkennung,
:wiki-Befehl, configurable Timeout/GPU/Context via ENV,
Streaming AbortController, Orphan-Codeblock-Detection)
This commit is contained in:
parent
f86c312870
commit
c1a984b22b
23 changed files with 718 additions and 95 deletions
BIN
.dreb/index/search.db
Normal file
BIN
.dreb/index/search.db
Normal file
Binary file not shown.
BIN
.dreb/index/search.db-shm
Normal file
BIN
.dreb/index/search.db-shm
Normal file
Binary file not shown.
BIN
.dreb/index/search.db-wal
Normal file
BIN
.dreb/index/search.db-wal
Normal file
Binary file not shown.
8
.dreb/memory/24b-compat.md
Normal file
8
.dreb/memory/24b-compat.md
Normal file
|
|
@ -0,0 +1,8 @@
|
||||||
|
### Änderungen am CLI (24B-Kompatibilität)
|
||||||
|
- **executeActions()** flexibler: erkennt `path:`, `Path:`, `File:`, `Datei:` zusätzlich zu `FILE:`
|
||||||
|
- **Orphan-Detektion**: Codeblöcke ohne Dateipfad (24B-typisch) werden erkannt und nach Pfad gefragt
|
||||||
|
- **`:wiki` Befehl**: Chunked Wiki-Ingest — eine Seite pro Request (für langsame Modelle)
|
||||||
|
- **Konfigurierbare Timeouts** via `LLAMA_TIMEOUT` (ENV)
|
||||||
|
- **Konfigurierbare GPU-Layers** via `LLAMA_GPU_LAYERS` (ENV)
|
||||||
|
- **Konfigurierbarer Kontext** via `LLAMA_CTX_SIZE` (ENV)
|
||||||
|
- **Streaming AbortController**: sauberer Timeout statt hängender Requests
|
||||||
158
MODELS.md
Normal file
158
MODELS.md
Normal file
|
|
@ -0,0 +1,158 @@
|
||||||
|
# MiniLlama — Modell-Testbericht
|
||||||
|
|
||||||
|
> Stand: 2026-07-09, getestet auf **Ryzen 5 2500U** (14 GB RAM, 4,7 GB Swap, CPU-only)
|
||||||
|
> Testumgebung: llama.cpp b9936, 4 Threads, ctx-size 4096, Temperatur 0.1
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🏆 Testsieger: LFM2-8B-A1B-UD-Q4_K_XL
|
||||||
|
|
||||||
|
| Eigenschaft | Wert |
|
||||||
|
|---|---|
|
||||||
|
| **Datei** | `LFM2-8B-A1B-UD-Q4_K_XL.gguf` |
|
||||||
|
| **Größe** | 4,5 GB |
|
||||||
|
| **Architektur** | MoE (1,5B aktiv / 8,5B total) |
|
||||||
|
| **Quelle** | [hugging-quants/LFM-8B-A1B](https://huggingface.co/hugging-quants/LFM-8B-A1B) |
|
||||||
|
| **IFEval** | 79,44 |
|
||||||
|
| **Ladezeit** | 12s (SSD) |
|
||||||
|
| **Prompt-Processing** | ~29 t/s |
|
||||||
|
| **Generation** | ~16,2 t/s |
|
||||||
|
| **RAM-Nutzung** | ~5 GB |
|
||||||
|
|
||||||
|
### ✅ Wiki-Test (IFTMIN für OBI)
|
||||||
|
- **FILE:-Pattern:** Immer korrekt
|
||||||
|
- **3 Wiki-Seiten:** Alle erstellt
|
||||||
|
- **Struktur:** Sehr gut (Tabellen, Überschriften, Codeblöcke)
|
||||||
|
- **Inhalt:** Fakten treu, gut formatiert
|
||||||
|
|
||||||
|
### ⚠️ Bekannte Limitierungen
|
||||||
|
- Keine
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## 🥇 Qualitätssieger: LFM2-24B-A2B-Q3_K_XL
|
||||||
|
|
||||||
|
| Eigenschaft | Wert |
|
||||||
|
|---|---|
|
||||||
|
| **Datei** | `LiquidAI_LFM2-24B-A2B-Q3_K_XL.gguf` |
|
||||||
|
| **Größe** | 11 GB |
|
||||||
|
| **Architektur** | MoE (8B aktiv / 24B total) |
|
||||||
|
| **Quelle** | [hugging-quants/LFM-24B-A2B](https://huggingface.co/hugging-quants/LFM-24B-A2B) |
|
||||||
|
| **IFEval** | ~85 (geschätzt, A2B-Variante) |
|
||||||
|
| **Ladezeit** | 14,6s (SSD) |
|
||||||
|
| **Prompt-Processing** | ~17 t/s |
|
||||||
|
| **Generation** | ~11,4 t/s |
|
||||||
|
| **RAM-Nutzung** | ~11 GB (mit mmap, nutzt Swap) |
|
||||||
|
|
||||||
|
### ✅ Wiki-Test (IFTMIN für OBI)
|
||||||
|
- **FILE:-Pattern:** Korrekt (alle 3 Dateien)
|
||||||
|
- **3 Wiki-Seiten:** Alle erstellt
|
||||||
|
- **Struktur:** Sehr gut
|
||||||
|
- **Inhalt:** Tiefere Einsichten und präzisere Zusammenhänge als 8B-Modell
|
||||||
|
- **Nuancen:** Deutlich bessere Unterscheidung zwischen Classic EDI und Web-IFTMIN
|
||||||
|
|
||||||
|
### ⚠️ Bekannte Limitierungen
|
||||||
|
- Braucht ~11 GB RAM + Swap → läuft, aber eng auf 14-GB-Maschine
|
||||||
|
- Auf Zielmaschine mit besserer GPU empfohlen
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ❌ Getestet — Nicht empfohlen
|
||||||
|
|
||||||
|
### LFM2-8B-A1B-IQ4_XS (4,2 GB)
|
||||||
|
| Test | Ergebnis |
|
||||||
|
|---|---|
|
||||||
|
| Ladezeit | 6s |
|
||||||
|
| Speed | 16,4 t/s |
|
||||||
|
| FILE:-Pattern | ❌ **Fehlt komplett** |
|
||||||
|
| Wiki-Qualität | ⚠️ Nur 2 von 3 Seiten |
|
||||||
|
| **Urteil** | Quantisierung zu aggressiv — Instruction Following bricht ein |
|
||||||
|
|
||||||
|
### LFM2-8B-A1B-Q3_K_XL (3,8 GB)
|
||||||
|
| Test | Ergebnis |
|
||||||
|
|---|---|
|
||||||
|
| Ladezeit | 6s |
|
||||||
|
| Speed | 15,3 t/s |
|
||||||
|
| FILE:-Pattern | ❌ **Fehlt oft** |
|
||||||
|
| Wiki-Qualität | ⚠️ Schwächer, unvollständig |
|
||||||
|
| **Urteil** | Q3-K quantisierung verliert zu viel Qualität |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## ✅ Früher getestet
|
||||||
|
|
||||||
|
| Modell | Größe | Speed | FILE:-Pattern | Status |
|
||||||
|
|---|---|---|---|---|
|
||||||
|
| Gemma 3 4B IT Q4_K_M | 2,4 GB | ~25 t/s | ✅ | Läuft auf Stick |
|
||||||
|
| Phi-4-mini Q4_K_M | 2,5 GB | ~22 t/s | ✅ | Läuft auf Stick |
|
||||||
|
| Qwen3.5-4B Q5_K_M | 3,0 GB | ~18 t/s | ✅ | Läuft auf Stick |
|
||||||
|
| Gemma-4-E2B Q4_K_M | 2,9 GB | ~20 t/s | ? | Läuft auf Stick |
|
||||||
|
| Qwen2.5-1.5B Q4_K_M | 1,0 GB | ~30 t/s | ❌ | Zu schwach |
|
||||||
|
| MiniCPM5-1B Q4_K_M | 688 MB | - | ❌ | Inkompatibel (Chat-Template) |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Test-Setup
|
||||||
|
|
||||||
|
### Hardware
|
||||||
|
- **CPU:** AMD Ryzen 5 2500U (4 Kerne/8 Threads)
|
||||||
|
- **RAM:** 14 GB DDR4
|
||||||
|
- **Swap:** 4,7 GB auf SSD
|
||||||
|
- **GPU:** AMD Radeon Vega Mobile (shared Memory, 1 GB reserviert) — nicht genutzt
|
||||||
|
- **Speicher:** Samsung SSD (diverse)
|
||||||
|
|
||||||
|
### Software
|
||||||
|
- **llama.cpp:** b9936 (CPU-only, keine GPU-Beschleunigung)
|
||||||
|
- **Threads:** 4 (`-t 4`)
|
||||||
|
- **Kontext:** 4096 (`--ctx-size 4096`)
|
||||||
|
- **GPU-Layers:** 0 (`--n-gpu-layers 0`)
|
||||||
|
|
||||||
|
### Test-Prompt (Wiki-Test)
|
||||||
|
```text
|
||||||
|
System: You are a Wiki Builder. Output FILE: path on its own line,
|
||||||
|
then fenced code block with complete markdown content.
|
||||||
|
|
||||||
|
User: Create wiki pages about IFTMIN for OBI.
|
||||||
|
TOPIC 1: GLN & NVE — ...
|
||||||
|
TOPIC 2: Classic EDI vs Web-IFTMIN — ...
|
||||||
|
Create: FILE: wiki/inhalt.md, FILE: wiki/GLN-NVE-Grundlagen.md,
|
||||||
|
FILE: wiki/Classic-EDI-vs-Web-IFTMIN.md
|
||||||
|
```
|
||||||
|
|
||||||
|
### Erfolgskriterien
|
||||||
|
1. **FILE:-Pattern:** Wird `FILE:` auf eigener Zeile vor dem Codeblock ausgegeben?
|
||||||
|
2. **Vollständigkeit:** Werden alle angeforderten Dateien erstellt?
|
||||||
|
3. **Qualität:** Ist der Inhalt strukturiert, fakten-treu, nützlich?
|
||||||
|
4. **Geschwindigkeit:** Reicht die Token-Rate für flüssigen Chat (>8 t/s)?
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Vergleich 8B vs 24B für Wiki-Ingest (`:wiki` Befehl)
|
||||||
|
|
||||||
|
> Getestet im `:wiki`-Befehl (Chunked: eine Seite pro Request, ctx-size 8192)
|
||||||
|
|
||||||
|
| Kriterium | 8B Q4_K_XL 🏆 | 24B Q3_K_XL |
|
||||||
|
|-----------|-------------|--------------|
|
||||||
|
| **Dauer: 12 Wiki-Seiten** | **~22 Min** | ~50 Min (Timeout) |
|
||||||
|
| **Generation Speed** | 15-20 t/s | 8,5-10 t/s |
|
||||||
|
| **Prompt-Processing** | ~29 t/s | ~13 t/s |
|
||||||
|
| **FILE:-Pattern (Harness)** | ✅ Immer korrekt | ❌ Unzuverlässig |
|
||||||
|
| **"Siehe auch"-Verweise** | ✅ Automatisch (11/12) | ✅ Automatisch |
|
||||||
|
| **Inhaltstiefe** | ✅ Sehr gut | ✅ Etwas tiefer |
|
||||||
|
| **Fazit** | **Empfohlen für Harness** | Nur bei spez. Qualitätslücken |
|
||||||
|
|
||||||
|
**Empfehlung:** 8B als Standard im CLI. 24B nur gezielt einsetzen wenn 8B-Qualität nicht ausreicht.
|
||||||
|
|
||||||
|
## CLI Harness: `:wiki` Befehl
|
||||||
|
|
||||||
|
Der `:wiki` Befehl im CrowdCode CLI wurde erweitert:
|
||||||
|
- Liest `raw/` + `AGENTS.md` aus dem Workspace
|
||||||
|
- Erstellt Wiki-Seiten Chunked (eine pro Request) — verhindert Prompt-Overload
|
||||||
|
- Konfigurierbar via ENV: `LLAMA_CTX_SIZE`, `LLAMA_TIMEOUT`
|
||||||
|
- Flexible Pattern-Erkennung: `FILE:`, `path:`, `Datei:`, orphane Codeblöcke
|
||||||
|
- Funktioniert mit 8B (schnell) und 24B (langsam)
|
||||||
|
|
||||||
|
### Qualitätsstufen (Entscheidungsbaum)
|
||||||
|
1. **8B als Ersteller** → Qualität prüfen (schnell, ~22 Min für 12 Seiten)
|
||||||
|
2. **8B als Kritiker** → Bei Lücken: 8B reviewt eigene Arbeit (schnell)
|
||||||
|
3. **24B Feinschliff** → Nur wenn 8B+8B nicht reicht (langsam, ~50 Min)
|
||||||
25
README.md
25
README.md
|
|
@ -46,17 +46,22 @@ MiniLlama/
|
||||||
|
|
||||||
## Verfügbare Modelle
|
## Verfügbare Modelle
|
||||||
|
|
||||||
| Modell | Größe | Typ | IFEval | Lädt | FILE:-Pattern |
|
> Ausführliche Testergebnisse in [MODELS.md](MODELS.md) — getestet auf Ryzen 5 2500U (14 GB RAM, CPU-only)
|
||||||
|--------|-------|-----|--------|------|---------------|
|
|
||||||
| **LFM2.5-8B-A1B** ⭐ | ~5,4 GB | MoE 1,5B aktiv | **91,84** | ? | ✅ Tool-Calling |
|
|
||||||
| **Gemma 3 4B IT** ⭐ | 2,4 GB | Dicht | **90,2** | ~3s | ✅ Getestet |
|
|
||||||
| **Phi-4-mini** ⭐ | 2,5 GB | Dicht | ~88 | ~3s | ✅ Getestet |
|
|
||||||
| **LFM2-8B-A1B** | 4,5 GB | MoE 1,5B aktiv | 79,44 | ? | ✅ (erwartet) |
|
|
||||||
| **Qwen3.5-4B** | 3,0 GB | Dicht | ~85 | ~47s | ✅ Getestet |
|
|
||||||
| **Gemma-4-E2B** | 2,9 GB | Dicht | ~80 | ~30s | ? |
|
|
||||||
| **Qwen2.5-1.5B** (klein) | 1,0 GB | Dicht | ~70 | ~2s | ❌ Zu schwach |
|
|
||||||
|
|
||||||
⭐ = Neu empfohlen (Juli 2026)
|
| Modell | Größe | Speed | FILE:-Pattern | Wiki-Qualität | Status |
|
||||||
|
|--------|-------|-------|---------------|---------------|--------|
|
||||||
|
| **LFM2-24B-A2B-Q3_K_XL** 🏆 | 11 GB | 11,4 t/s | ⚠️ Manuell | ✅ Sehr gut (tief) | Qualitätssieger (langsam) |
|
||||||
|
| **LFM2-8B-A1B-UD-Q4_K_XL** 🏆 | 4,5 GB | 16,2 t/s | ✅ Immer | ✅ Sehr gut | **Testsieger/Default** |
|
||||||
|
| **LFM2.5-8B-A1B-Q4_K_M** ⭐ | 4,9 GB | ? | ? | ? | Noch zu testen |
|
||||||
|
| **Gemma 3 4B IT** ⭐ | 2,4 GB | ~25 t/s | ✅ | ✅ Gut | Leichtes Fallback |
|
||||||
|
| **Phi-4-mini** ⭐ | 2,5 GB | ~22 t/s | ✅ | ✅ Gut | Leichtes Fallback |
|
||||||
|
| **Qwen3.5-4B** | 3,0 GB | ~18 t/s | ✅ | ✅ Gut | Alternativ |
|
||||||
|
| **LFM2-8B-A1B-IQ4_XS** | 4,2 GB | 16,4 t/s | ❌ Fehlt | ⚠️ Mittel | **Nicht empfohlen** |
|
||||||
|
| **LFM2-8B-A1B-Q3_K_XL** | 3,8 GB | 15,3 t/s | ❌ Fehlt oft | ⚠️ Schwach | **Nicht empfohlen** |
|
||||||
|
| **Qwen2.5-1.5B** | 1,0 GB | ~30 t/s | ❌ | ❌ Zu schwach | Nur Notfall |
|
||||||
|
| **MiniCPM5-1B** | 688 MB | - | ❌ | ❌ Inkompatibel | ❌ |
|
||||||
|
|
||||||
|
⭐ = Noch zu testen · 🏆 = Getestet und empfohlen (Juli 2026)
|
||||||
|
|
||||||
## CrowdCode CLI Features
|
## CrowdCode CLI Features
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -37,6 +37,10 @@
|
||||||
- Qwen2.5-1.5B zu schwach für Instruction Following (FILE:-Pattern)
|
- Qwen2.5-1.5B zu schwach für Instruction Following (FILE:-Pattern)
|
||||||
- Gemma 3 4B IT und Phi-4-mini funktionieren perfekt (IFEval 90.2/88, laden in ~3s)
|
- Gemma 3 4B IT und Phi-4-mini funktionieren perfekt (IFEval 90.2/88, laden in ~3s)
|
||||||
- LFM2.5-8B-A1B hat beste IFEval (91.84), noch nicht getestet
|
- LFM2.5-8B-A1B hat beste IFEval (91.84), noch nicht getestet
|
||||||
|
- **LFM2-8B-A1B-UD-Q4_K_XL (4,5 GB): Testsieger** — 16,2 t/s, FILE:-Pattern ✅, Wiki sehr gut ✅
|
||||||
|
- **LFM2-24B-A2B-Q3_K_XL (11 GB): Qualitätssieger** — 11,4 t/s, FILE:-Pattern ✅, Wiki sehr gut ✅
|
||||||
|
- **LFM2-8B-A1B-IQ4_XS (4,2 GB):** ❌ FILE:-Pattern fehlt komplett (Quantisierung zu aggressiv)
|
||||||
|
- **LFM2-8B-A1B-Q3_K_XL (3,8 GB):** ❌ FILE:-Pattern fehlt oft (Q3 zu niedrig)
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -45,10 +49,11 @@
|
||||||
| # | Entscheidung | Begründung | Datum |
|
| # | Entscheidung | Begründung | Datum |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| 1 | CLI in Node.js statt Bash | Plattformunabhängig (Win+Lin), fetch, readline | 2026-07-09 |
|
| 1 | CLI in Node.js statt Bash | Plattformunabhängig (Win+Lin), fetch, readline | 2026-07-09 |
|
||||||
| 2 | Gemma 3 4B IT als Standard | IFEval 90.2, nur 2.4 GB, lädt in 3s | 2026-07-09 |
|
| 2 | LFM2-8B-A1B-UD-Q4_K_XL als Standard | 16,2 t/s, FILE:-Pattern ✅, Wiki sehr gut ✅, nur 4,5 GB | 2026-07-09 |
|
||||||
| 3 | Modelle als Symlinks auf Stick | Spart Platz im Repo, flexibler Austausch | 2026-07-09 |
|
| 3 | LFM2-24B-A2B-Q3_K_XL für Qualität reserviert | 11,4 t/s, tiefere Einsichten — für Zielmaschine mit GPU | 2026-07-09 |
|
||||||
| 4 | Server-Auto-Start im CLI | Kein manueller Vorstart nötig, Modellauswahl inklusive | 2026-07-09 |
|
| 4 | Modelle als Symlinks auf Stick | Spart Platz im Repo, flexibler Austausch | 2026-07-09 |
|
||||||
| 5 | Shell-Kommando-Erkennung | pwd/ls/git direkt ausführen, nicht ans LLM | 2026-07-09 |
|
| 5 | Server-Auto-Start im CLI | Kein manueller Vorstart nötig, Modellauswahl inklusive | 2026-07-09 |
|
||||||
|
| 6 | Shell-Kommando-Erkennung | pwd/ls/git direkt ausführen, nicht ans LLM | 2026-07-09 |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -56,12 +61,12 @@
|
||||||
|
|
||||||
| # | Aktion | Priorität | Status |
|
| # | Aktion | Priorität | Status |
|
||||||
|---|---|---|---|
|
|---|---|---|---|
|
||||||
| 1 | LFM2-8B-A1B-UD-Q4_K_XL.gguf auf dem Stick mit FILE:-Pattern testen | 1 (sofort) | ⬜ |
|
| 1 | LFM2.5-8B-A1B-Q4_K_M testen (IFEval 91.84) | 1 (sofort) | ⬜ |
|
||||||
| 2 | LFM2.5-8B-A1B-Q4_K_M.gguf downloaden und testen (IFEval 91.84) | 1 (sofort) | ⬜ |
|
| 2 | CrowdCode VSCode Extension als .vsix deployen und testen | 2 (danach) | ⬜ |
|
||||||
| 3 | CrowdCode VSCode Extension als .vsix deployen und testen | 2 (danach) | ⬜ |
|
| 3 | Wiki-Modus mit iftmin-wiki testen | 2 (danach) | ⬜ |
|
||||||
| 4 | Wiki-Modus mit iftmin-wiki testen | 2 (danach) | ⬜ |
|
| 4 | Windows-Test des CLI (crowdcode.bat) auf Zielrechner | 2 (danach) | ⬜ |
|
||||||
| 5 | Windows-Test des CLI (crowdcode.bat) auf Zielrechner | 2 (danach) | ⬜ |
|
| 5 | Performance-Benchmarks auf Zielrechner (Ladezeit, t/s) | 3 (optional) | ⬜ |
|
||||||
| 6 | Performance-Benchmarks auf Zielrechner (Ladezeit, t/s) | 3 (optional) | ⬜ |
|
| 6 | LFM2-24B auf Zielmaschine mit GPU testen | 3 (optional) | ⬜ |
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
|
@ -93,3 +98,33 @@
|
||||||
1. LFM2-8B-A1B auf Stick testen (der liegt schon da)
|
1. LFM2-8B-A1B auf Stick testen (der liegt schon da)
|
||||||
2. LFM2.5-8B-A1B downloaden
|
2. LFM2.5-8B-A1B downloaden
|
||||||
3. Windows-Kompatibilität des CLI prüfen
|
3. Windows-Kompatibilität des CLI prüfen
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Session: 2026-07-09 14:30-18:00
|
||||||
|
|
||||||
|
### Erreichtes
|
||||||
|
- **24B Wiki-Test:** 11 Seiten in 12 Min (manuell, eine Seite pro Request)
|
||||||
|
- **24B Harness-Test:** 3 Seiten in 20 Min via :wiki (zu langsam, ctx-Probleme gefixt)
|
||||||
|
- **8B Wiki-Test:** 12 Seiten in 22 Min via :wiki + 2 Min Nacharbeit
|
||||||
|
- **CLI-Harness erweitert:**
|
||||||
|
- :wiki Befehl für Chunked Wiki-Ingest (eine Seite pro Request)
|
||||||
|
- Flexible FILE:-Pattern-Erkennung (path:, File:, Datei:, orphane Codeblöcke)
|
||||||
|
- Konfigurierbare Timeouts/GPU/Context-Size via ENV
|
||||||
|
- Streaming AbortController für saubere Timeouts
|
||||||
|
- **Iftmin-Wiki:** 12 Wiki-Seiten, 76 KB, alle mit Querverweisen ✅
|
||||||
|
- **MODELS.md:** Um 8B vs 24B Vergleich + :wiki-Doku erweitert
|
||||||
|
- **Backup:** Backup als tar.gz erstellt (50 MB)
|
||||||
|
- **CrowdBrain:** minillama.md mit Testergebnissen aktualisiert
|
||||||
|
|
||||||
|
### Entscheidungen
|
||||||
|
- **8B als Default im Harness** — schnell (16-20 t/s), FILE:-Pattern ✅, Wiki ✅
|
||||||
|
- **24B nur bei Qualitätslücken** — langsam (8-10 t/s), FILE:-Pattern unzuverlässig
|
||||||
|
- **Qualitäts-Pipeline:** 8B erstellen → 8B reviewen → 24B nur wenn nötig
|
||||||
|
- **Modelle unter 10 t/s + aussortiert → entfernt** (MiniCPM5-1B und andere)
|
||||||
|
|
||||||
|
### Nächste Schritte
|
||||||
|
1. Windows-Test des CLI auf Zielrechner
|
||||||
|
2. LFM2.5-8B-A1B testen (IFEval 91.84, noch nicht evaluiert)
|
||||||
|
3. 24B auf Zielmaschine mit GPU testen
|
||||||
|
4. Wiki-Git-Repo anlegen (iftmin-wiki)
|
||||||
|
|
|
||||||
168
TEST_SCENARIO.md
Normal file
168
TEST_SCENARIO.md
Normal file
|
|
@ -0,0 +1,168 @@
|
||||||
|
# MiniLlama — CLI Test-Szenario
|
||||||
|
|
||||||
|
> Dieses Szenario testet die Kernfunktionen des CrowdCode CLI:
|
||||||
|
> Datei-Operationen (FILE:, DIR:, EDIT:), Shell-Zugriff (```bash),
|
||||||
|
> @mentions, und Chat.
|
||||||
|
|
||||||
|
## Vorbereitung
|
||||||
|
|
||||||
|
Sicherstellen, dass der Server läuft oder über den CLI automatisch gestartet wird:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /home/xray/code/MiniLlama
|
||||||
|
./start.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Oder für API-Zugriff:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd /home/xray/code/MiniLlama
|
||||||
|
./start-server.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
## Test 1: Einfacher Chat
|
||||||
|
|
||||||
|
**Eingabe:**
|
||||||
|
```
|
||||||
|
Sag kurz: Hallo Welt
|
||||||
|
```
|
||||||
|
|
||||||
|
**Erwartet:** Antwort "Hallo Welt!" (oder ähnlich)
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Test 2: Verzeichnis + Datei anlegen
|
||||||
|
|
||||||
|
**Eingabe:**
|
||||||
|
```
|
||||||
|
Lege ein Verzeichnis test_reihe an, erstelle darin Test.txt mit dem Text "ich bin ein test"
|
||||||
|
```
|
||||||
|
|
||||||
|
**Erwartet:**
|
||||||
|
- CLI zeigt `DIR: test_reihe` → Verzeichnis wird erstellt
|
||||||
|
- CLI zeigt `FILE: test_reihe/Test.txt` → Datei wird geschrieben
|
||||||
|
- Inhalt: `ich bin ein test`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Test 3: Datei kopieren
|
||||||
|
|
||||||
|
**Eingabe:**
|
||||||
|
```
|
||||||
|
kopiere Test.txt als Test2.txt ins gleiche Verzeichnis
|
||||||
|
```
|
||||||
|
|
||||||
|
**Erwartet:**
|
||||||
|
- CLI zeigt `` ```bash cp test_reihe/Test.txt test_reihe/Test2.txt````
|
||||||
|
- CLI fragt: "Ausführen? (j/N)" → **j** eingeben
|
||||||
|
- `Test2.txt` existiert mit gleichem Inhalt
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Test 4: Shell-Befehl (direkt)
|
||||||
|
|
||||||
|
**Eingabe:**
|
||||||
|
```
|
||||||
|
ls -la test_reihe/
|
||||||
|
```
|
||||||
|
|
||||||
|
**Erwartet:**
|
||||||
|
- Wird direkt als Shell-Befehl erkannt (kein LLM)
|
||||||
|
- Zeigt `Test.txt` und `Test2.txt`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Test 5: Shell-Befehl mit Prompt-Präfix
|
||||||
|
|
||||||
|
**Eingabe:**
|
||||||
|
```
|
||||||
|
$ pwd
|
||||||
|
```
|
||||||
|
|
||||||
|
**Erwartet:**
|
||||||
|
- `$ ` wird ignoriert, `pwd` wird erkannt
|
||||||
|
- Zeigt aktuellen Pfad
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Test 6: Datei lesen (@mention)
|
||||||
|
|
||||||
|
**Eingabe:**
|
||||||
|
```
|
||||||
|
@test_reihe/Test.txt
|
||||||
|
```
|
||||||
|
|
||||||
|
**Erwartet:**
|
||||||
|
- Inhalt `ich bin ein test` wird eingelesen und ans LLM übergeben
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Test 7: Verzeichnis auflisten (@mention)
|
||||||
|
|
||||||
|
**Eingabe:**
|
||||||
|
```
|
||||||
|
@test_reihe/
|
||||||
|
```
|
||||||
|
|
||||||
|
**Erwartet:**
|
||||||
|
- Listet `Test.txt` und `Test2.txt`
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Test 8: Natürlichsprachliche Dateiabfrage
|
||||||
|
|
||||||
|
**Eingabe:**
|
||||||
|
```
|
||||||
|
liste die dateien in test_reihe auf
|
||||||
|
```
|
||||||
|
|
||||||
|
**Erwartet:**
|
||||||
|
- LLM gibt `` ```bash ls -la test_reihe/```` zurück
|
||||||
|
- CLI fragt "Ausführen? (j/N)" → **j** eingeben
|
||||||
|
- Zeigt Dateiliste
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Test 9: Mehrere Dateien auf einmal (Wiki-Stil)
|
||||||
|
|
||||||
|
**Eingabe:**
|
||||||
|
```
|
||||||
|
Erstelle folgende Dateien:
|
||||||
|
- DIR: projekt/src
|
||||||
|
- FILE: projekt/src/hello.py mit einem print("Hallo Welt")
|
||||||
|
- FILE: projekt/README.md mit einer kurzen Beschreibung
|
||||||
|
```
|
||||||
|
|
||||||
|
**Erwartet:**
|
||||||
|
- Alle DIR:- und FILE:-Patterns werden automatisch ausgeführt
|
||||||
|
- Keine Bestätigungsabfragen nötig
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Test 10: Aufräumen
|
||||||
|
|
||||||
|
**Eingabe:**
|
||||||
|
```
|
||||||
|
Lösche das test_reihe und projekt Verzeichnis
|
||||||
|
```
|
||||||
|
|
||||||
|
**Erwartet:**
|
||||||
|
- CLI zeigt `` ```bash rm -rf test_reihe projekt````
|
||||||
|
- CLI fragt "Ausführen? (j/N)" → **j** für Bestätigung
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Erfolgskriterien
|
||||||
|
|
||||||
|
| Test | Feature | Status |
|
||||||
|
|---|---|---|
|
||||||
|
| 1 | Chat | ⬜ |
|
||||||
|
| 2 | DIR: + FILE: | ⬜ |
|
||||||
|
| 3 | ```bash (cp) mit Bestätigung | ⬜ |
|
||||||
|
| 4 | Shell direkt | ⬜ |
|
||||||
|
| 5 | Prompt-Präfix $ | ⬜ |
|
||||||
|
| 6 | @mention (lesen) | ⬜ |
|
||||||
|
| 7 | @mention (dir) | ⬜ |
|
||||||
|
| 8 | NL-Dateiabfrage via LLM | ⬜ |
|
||||||
|
| 9 | Multi-File Wiki-Stil | ⬜ |
|
||||||
|
| 10 | Aufräumen | ⬜ |
|
||||||
307
cli/crowdcode.js
307
cli/crowdcode.js
|
|
@ -3,9 +3,14 @@
|
||||||
// Chat + Dateioperationen via llama.cpp API
|
// Chat + Dateioperationen via llama.cpp API
|
||||||
// Aufruf: node crowdcode.js [endpoint] [model]
|
// Aufruf: node crowdcode.js [endpoint] [model]
|
||||||
|
|
||||||
// ─── Konfiguration ────────────────────────────────────────────────────────────
|
// ─── Konfiguration (via ENV überschreibbar) ──────────────────────────────────
|
||||||
const API = process.argv[2] || 'http://127.0.0.1:8080/v1';
|
const API = process.argv[2] || process.env.LLAMA_API || 'http://127.0.0.1:8080/v1';
|
||||||
const MODEL = process.argv[3] || '';
|
const MODEL = process.argv[3] || process.env.LLAMA_MODEL || '';
|
||||||
|
const CONFIG = {
|
||||||
|
timeout: parseInt(process.env.LLAMA_TIMEOUT || '120000'), // Streaming-Timeout (ms)
|
||||||
|
ctxSize: parseInt(process.env.LLAMA_CTX_SIZE || '4096'), // Kontext-Fenster
|
||||||
|
gpuLayers: parseInt(process.env.LLAMA_GPU_LAYERS || '0'), // GPU-Layer für --n-gpu-layers
|
||||||
|
};
|
||||||
|
|
||||||
// ─── Terminal-Farben (ANSI, funktioniert auf Win/Lin) ─────────────────────────
|
// ─── Terminal-Farben (ANSI, funktioniert auf Win/Lin) ─────────────────────────
|
||||||
const C = {
|
const C = {
|
||||||
|
|
@ -29,14 +34,24 @@ const { execSync } = require('child_process');
|
||||||
|
|
||||||
const WORKSPACE = process.argv[4] ? path.resolve(process.argv[4]) : process.cwd();
|
const WORKSPACE = process.argv[4] ? path.resolve(process.argv[4]) : process.cwd();
|
||||||
|
|
||||||
const SYSTEM_PROMPT = `You are CrowdCode, a concise coding assistant.
|
const SYSTEM_PROMPT = `You are CrowdCode, a concise coding assistant with FULL shell access.
|
||||||
Context: @file reads a file, @dir/ lists a directory, @dir/** shows the tree, @?term searches in files.
|
|
||||||
When writing or rewriting a file, ALWAYS output FILE: path/to/file on its own line BEFORE the fenced code block with the complete file content.
|
FILE OPERATIONS (no confirmation needed):
|
||||||
To create a directory: DIR: path/to/dir on its own line.
|
- DIR: path/to/dir — create a directory
|
||||||
For targeted edits: EDIT: path/to/file followed by ---OLD, old text, ---NEW, new text, --- on separate lines.
|
- FILE: path/to/file on its own line, then fenced code block with COMPLETE file content — create or overwrite a file
|
||||||
To run shell commands (only when user explicitly asks): output a fenced \`\`\`bash code block.
|
- EDIT: path/to/file, then ---OLD, old text, ---NEW, new text, --- — edit a file
|
||||||
No explanations unless the user asks a question.
|
|
||||||
NEVER output touch, mkdir, echo, cat commands — use FILE: and DIR: instead.`;
|
SHELL COMMANDS (user confirms each):
|
||||||
|
- Use \`\`\`bash for: cp, mv, rm, grep, find, git, npm, and other shell operations that aren't file/dir creation
|
||||||
|
- When the user asks about files/dirs/system, use a \`\`\`bash block
|
||||||
|
|
||||||
|
Examples:
|
||||||
|
- user: "erstelle test.txt mit Hallo" → DIR: tests \n FILE: tests/test.txt \n\`\`\`\nHallo\n\`\`\`
|
||||||
|
- user: "liste dateien" → \`\`\`bash\nls -la\n\`\`\`
|
||||||
|
- user: "kopiere test.txt als test2.txt" → \`\`\`bash\ncp test.txt test2.txt\n\`\`\`
|
||||||
|
|
||||||
|
Never explain — just output the commands.
|
||||||
|
NEVER use echo, touch, mkdir, cat, printf to create files — use FILE: and DIR: instead.`;
|
||||||
|
|
||||||
function askSingleLine(prompt) {
|
function askSingleLine(prompt) {
|
||||||
const rl = readline.createInterface({ input: process.stdin, output: process.stdout });
|
const rl = readline.createInterface({ input: process.stdin, output: process.stdout });
|
||||||
|
|
@ -80,43 +95,52 @@ async function fetchModels() {
|
||||||
}
|
}
|
||||||
|
|
||||||
// ─── Chat Completion (Streaming) ───────────────────────────────────────────────
|
// ─── Chat Completion (Streaming) ───────────────────────────────────────────────
|
||||||
async function chatComplete(messages, onToken) {
|
async function chatComplete(messages, onToken, maxTokens) {
|
||||||
const model = MODEL || (await fetchModels())[0]?.id || 'unknown';
|
const model = MODEL || (await fetchModels())[0]?.id || 'unknown';
|
||||||
const res = await fetch(`${API}/chat/completions`, {
|
const controller = new AbortController();
|
||||||
method: 'POST',
|
const timeoutId = setTimeout(() => controller.abort(), CONFIG.timeout);
|
||||||
headers: { 'Content-Type': 'application/json', 'Authorization': 'Bearer local' },
|
try {
|
||||||
body: JSON.stringify({ model, messages, stream: true, temperature: 0.2, top_p: 0.9 }),
|
const body = JSON.stringify({ model, messages, stream: true, temperature: 0.2, top_p: 0.9, ...(maxTokens ? { max_tokens: maxTokens } : {}) });
|
||||||
});
|
const res = await fetch(`${API}/chat/completions`, {
|
||||||
if (!res.ok || !res.body) throw new Error(`HTTP ${res.status}`);
|
method: 'POST',
|
||||||
const reader = res.body.getReader();
|
headers: { 'Content-Type': 'application/json', 'Authorization': 'Bearer local' },
|
||||||
const decoder = new TextDecoder();
|
body,
|
||||||
let fullText = '';
|
signal: controller.signal,
|
||||||
while (true) {
|
});
|
||||||
const { done, value } = await reader.read();
|
clearTimeout(timeoutId);
|
||||||
if (done) break;
|
if (!res.ok || !res.body) throw new Error(`HTTP ${res.status}`);
|
||||||
const chunk = decoder.decode(value, { stream: true });
|
const reader = res.body.getReader();
|
||||||
for (const line of chunk.split('\n')) {
|
const decoder = new TextDecoder();
|
||||||
const trimmed = line.trim();
|
let fullText = '';
|
||||||
if (!trimmed.startsWith('data:')) continue;
|
while (true) {
|
||||||
const data = trimmed.slice(5).trim();
|
const { done, value } = await reader.read();
|
||||||
if (data === '[DONE]') continue;
|
if (done) break;
|
||||||
try {
|
const chunk = decoder.decode(value, { stream: true });
|
||||||
const parsed = JSON.parse(data);
|
for (const line of chunk.split('\n')) {
|
||||||
// reasoning_content für DeepSeek-artige Modelle
|
const trimmed = line.trim();
|
||||||
const reasoning = parsed.choices?.[0]?.delta?.reasoning_content || '';
|
if (!trimmed.startsWith('data:')) continue;
|
||||||
const token = parsed.choices?.[0]?.delta?.content || '';
|
const data = trimmed.slice(5).trim();
|
||||||
if (reasoning) {
|
if (data === '[DONE]') continue;
|
||||||
fullText += reasoning;
|
try {
|
||||||
onToken(reasoning, true);
|
const parsed = JSON.parse(data);
|
||||||
}
|
// reasoning_content für DeepSeek-artige Modelle
|
||||||
if (token) {
|
const reasoning = parsed.choices?.[0]?.delta?.reasoning_content || '';
|
||||||
fullText += token;
|
const token = parsed.choices?.[0]?.delta?.content || '';
|
||||||
onToken(token, false);
|
if (reasoning) {
|
||||||
}
|
fullText += reasoning;
|
||||||
} catch { /* skip malformed JSON */ }
|
onToken(reasoning, true);
|
||||||
|
}
|
||||||
|
if (token) {
|
||||||
|
fullText += token;
|
||||||
|
onToken(token, false);
|
||||||
|
}
|
||||||
|
} catch { /* skip malformed JSON */ }
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
return fullText;
|
||||||
|
} finally {
|
||||||
|
clearTimeout(timeoutId);
|
||||||
}
|
}
|
||||||
return fullText;
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// ─── @mention Auflösung ───────────────────────────────────────────────────────
|
// ─── @mention Auflösung ───────────────────────────────────────────────────────
|
||||||
|
|
@ -231,31 +255,59 @@ function walkSync(dir) {
|
||||||
function executeActions(response) {
|
function executeActions(response) {
|
||||||
const actions = [];
|
const actions = [];
|
||||||
|
|
||||||
// FILE: pattern
|
// === FILE:/path: Pattern (8B + 24B kompatibel) ===
|
||||||
const fileRegex = /FILE:\s*(.+)\n```(?:\w+)?\n([\s\S]+?)```/g;
|
// 8B: FILE: pfad\n```\n...```
|
||||||
|
// 24B: path: name.md\n```markdown\n...```
|
||||||
|
// 24B: Path: name.md\n```\n...```
|
||||||
|
const fileWithPrefixRegex = /(?:^|\n)[ \t]*(?:FILE|Path|path|Datei|datei|file|File)[:\s]\s*`?\s*(.+?)\s*`?\n```(?:\w+)?\n([\s\S]+?)```/g;
|
||||||
let m;
|
let m;
|
||||||
while ((m = fileRegex.exec(response)) !== null) {
|
const matchedRanges = [];
|
||||||
actions.push({ type: 'file', path: m[1].trim(), content: m[2] });
|
|
||||||
|
while ((m = fileWithPrefixRegex.exec(response)) !== null) {
|
||||||
|
let filePath = m[1].trim().replace(/^`+|`+$/g, '');
|
||||||
|
if (filePath && filePath.length < 300) {
|
||||||
|
actions.push({ type: 'file', path: filePath, content: m[2].trim() });
|
||||||
|
matchedRanges.push({ start: m.index, end: m.index + m[0].length });
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// DIR: pattern
|
// === DIR: pattern ===
|
||||||
const dirRegex = /^DIR:\s*(.+)$/gm;
|
const dirRegex = /^DIR:\s*(.+)$/gm;
|
||||||
while ((m = dirRegex.exec(response)) !== null) {
|
while ((m = dirRegex.exec(response)) !== null) {
|
||||||
actions.push({ type: 'dir', path: m[1].trim() });
|
actions.push({ type: 'dir', path: m[1].trim() });
|
||||||
}
|
}
|
||||||
|
|
||||||
// EDIT: pattern
|
// === EDIT: pattern ===
|
||||||
const editRegex = /EDIT:\s*(.+?)\n---OLD\n([\s\S]*?)\n---NEW\n([\s\S]*?)\n---/gm;
|
const editRegex = /EDIT:\s*(.+?)\n---OLD\n([\s\S]*?)\n---NEW\n([\s\S]*?)\n---/gm;
|
||||||
while ((m = editRegex.exec(response)) !== null) {
|
while ((m = editRegex.exec(response)) !== null) {
|
||||||
actions.push({ type: 'edit', path: m[1].trim(), oldText: m[2], newText: m[3] });
|
actions.push({ type: 'edit', path: m[1].trim(), oldText: m[2], newText: m[3] });
|
||||||
}
|
}
|
||||||
|
|
||||||
// Bash pattern
|
// === Bash pattern ===
|
||||||
const bashRegex = /```bash\n([\s\S]+?)```/g;
|
const bashRegex = /```bash\n([\s\S]+?)```/g;
|
||||||
while ((m = bashRegex.exec(response)) !== null) {
|
while ((m = bashRegex.exec(response)) !== null) {
|
||||||
actions.push({ type: 'bash', command: m[1].trim() });
|
actions.push({ type: 'bash', command: m[1].trim() });
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// === Namenlose Markdown-Codeblöcke (24B-Fallback) ===
|
||||||
|
// Erfasst Codeblöcke ohne FILE:-Prefix, die wie Wiki-Seiten aussehen
|
||||||
|
const orphanCodeRegex = /```(?:\w+)?\n([\s\S]+?)```/g;
|
||||||
|
while ((m = orphanCodeRegex.exec(response)) !== null) {
|
||||||
|
// Bereits abgedeckt?
|
||||||
|
const alreadyMatched = matchedRanges.some(r => m.index >= r.start && m.index < r.end);
|
||||||
|
if (alreadyMatched) continue;
|
||||||
|
|
||||||
|
const content = m[1].trim();
|
||||||
|
// Nur Markdown-ähnliche Inhalte mit Substanz erkennen
|
||||||
|
if (content.length > 50 &&
|
||||||
|
(content.startsWith('#') || content.includes('\n##') ||
|
||||||
|
content.includes('|') || content.includes('Siehe auch') ||
|
||||||
|
content.includes('Gliederung') || content.includes('Inhalt') ||
|
||||||
|
content.includes('Übersicht') || content.includes('Grundlagen'))) {
|
||||||
|
actions.push({ type: 'file-orphan', content });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
return actions;
|
return actions;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -270,6 +322,22 @@ async function applyActions(actions) {
|
||||||
console.log(` ${C.green}✓${C.reset} Datei gespeichert: ${path.relative(WORKSPACE, fullPath)}`);
|
console.log(` ${C.green}✓${C.reset} Datei gespeichert: ${path.relative(WORKSPACE, fullPath)}`);
|
||||||
break;
|
break;
|
||||||
}
|
}
|
||||||
|
case 'file-orphan': {
|
||||||
|
// Namenloser Codeblock — User nach Pfad fragen
|
||||||
|
const firstLine = a.content.split('\n')[0].trim().replace(/^#+\s*/, '');
|
||||||
|
console.log(` ${C.cyan}📄${C.reset} Codeblock erkannt: ${C.grey}"${firstLine}"${C.reset}`);
|
||||||
|
const pfad = await askSingleLine(` ${C.yellow}Speichern als?${C.reset} (wiki/Dateiname.md / Enter=überspringen) `);
|
||||||
|
if (pfad && pfad.trim()) {
|
||||||
|
const fullPath = path.resolve(WORKSPACE, pfad.trim());
|
||||||
|
const parentDir = path.dirname(fullPath);
|
||||||
|
fs.mkdirSync(parentDir, { recursive: true });
|
||||||
|
fs.writeFileSync(fullPath, a.content, 'utf8');
|
||||||
|
console.log(` ${C.green}✓${C.reset} Datei gespeichert: ${path.relative(WORKSPACE, fullPath)}`);
|
||||||
|
} else {
|
||||||
|
console.log(` ${C.dim} (übersprungen)${C.reset}`);
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
}
|
||||||
case 'dir': {
|
case 'dir': {
|
||||||
const fullPath = path.resolve(WORKSPACE, a.path);
|
const fullPath = path.resolve(WORKSPACE, a.path);
|
||||||
fs.mkdirSync(fullPath, { recursive: true });
|
fs.mkdirSync(fullPath, { recursive: true });
|
||||||
|
|
@ -386,8 +454,8 @@ async function startServerWithModel(localModels) {
|
||||||
'-m', modelPath.path,
|
'-m', modelPath.path,
|
||||||
'--host', '127.0.0.1',
|
'--host', '127.0.0.1',
|
||||||
'--port', '8080',
|
'--port', '8080',
|
||||||
'--ctx-size', '4096',
|
'--ctx-size', String(CONFIG.ctxSize),
|
||||||
'--n-gpu-layers', '0',
|
'--n-gpu-layers', String(CONFIG.gpuLayers),
|
||||||
], {
|
], {
|
||||||
cwd: MINILLAMA_DIR,
|
cwd: MINILLAMA_DIR,
|
||||||
stdio: ['ignore', 'pipe', 'pipe'],
|
stdio: ['ignore', 'pipe', 'pipe'],
|
||||||
|
|
@ -429,7 +497,116 @@ async function startServerWithModel(localModels) {
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
// ─── Hauptprogramm ────────────────────────────────────────────────────────────
|
// ─── Wiki-Ingest (Chunked: eine Seite pro Request) ──────────────────────────
|
||||||
|
async function wikiIngest(chatCompleteFn) {
|
||||||
|
const wikiDir = path.join(WORKSPACE, 'wiki');
|
||||||
|
const rawDir = path.join(WORKSPACE, 'raw');
|
||||||
|
const agentsFile = path.join(WORKSPACE, 'AGENTS.md');
|
||||||
|
|
||||||
|
// Prüfe ob Wiki-Umgebung
|
||||||
|
if (!fs.existsSync(rawDir) || !fs.existsSync(agentsFile)) {
|
||||||
|
console.log(` ${C.red}❌${C.reset} Keine Wiki-Umgebung gefunden.`);
|
||||||
|
console.log(` ${C.dim} Erwartet: raw/ (Quellen) und AGENTS.md (Konfiguration) im Workspace.${C.reset}`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
const rawFiles = fs.readdirSync(rawDir).filter(f => f.endsWith('.md'));
|
||||||
|
if (rawFiles.length === 0) {
|
||||||
|
console.log(` ${C.red}❌${C.reset} Keine .md-Quellen in raw/ gefunden.`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// AGENTS.md lesen
|
||||||
|
const agents = fs.readFileSync(agentsFile, 'utf8');
|
||||||
|
const assistantsFile = path.join(WORKSPACE, 'wiki-assistant.md');
|
||||||
|
const assistants = fs.existsSync(assistantsFile) ? fs.readFileSync(assistantsFile, 'utf8') : '';
|
||||||
|
|
||||||
|
// Rohdaten lesen
|
||||||
|
const rawContents = {};
|
||||||
|
const rawCompact = {};
|
||||||
|
for (const f of rawFiles) {
|
||||||
|
const content = fs.readFileSync(path.join(rawDir, f), 'utf8');
|
||||||
|
rawContents[f] = content;
|
||||||
|
// Kompakte Version: erste 800 Zeichen reichen als Kontext
|
||||||
|
rawCompact[f] = (content.length > 800 ? content.substring(0, 800) + '\n...' : content);
|
||||||
|
}
|
||||||
|
// Kompakte Gesamt-Rohdaten (max ~1600 Zeichen für alle Quellen)
|
||||||
|
const compactSources = Object.entries(rawCompact).map(([name, c]) => `--- ${name} ---\n${c}`).join('\n\n');
|
||||||
|
const fullSources = Object.entries(rawContents).map(([name, c]) => `--- ${name} ---\n${c}`).join('\n\n');
|
||||||
|
|
||||||
|
// Themen aus AGENTS.md extrahieren
|
||||||
|
const topicMatch = agents.match(/## Aktuelle Wiki-Themen\n\n([\s\S]*?)(?:\n##|\n$)/);
|
||||||
|
const topics = [];
|
||||||
|
if (topicMatch) {
|
||||||
|
const lines = topicMatch[1].split('\n').filter(l => l.trim().startsWith('- **'));
|
||||||
|
for (const line of lines) {
|
||||||
|
const m = line.match(/- \*\*(.+?)\*\*\s*[\u2014\u2013-]\s*(.+)/);
|
||||||
|
if (m) topics.push({ title: m[1].trim(), description: m[2].trim() });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (topics.length === 0) {
|
||||||
|
console.log(` ${C.yellow}⚠️${C.reset} Keine Themen in AGENTS.md gefunden.`);
|
||||||
|
console.log(` ${C.dim} Bitte pflege "## Aktuelle Wiki-Themen" in AGENTS.md.${C.reset}`);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
// Inhalt.md als erstes — Gesamtüberblick
|
||||||
|
const pages = [
|
||||||
|
{ file: 'Inhalt.md', title: 'Inhalt (Startseite)', prompt: `Create wiki/Inhalt.md — the index page for this wiki about IFTMIN (EDIFACT Transportauftrag) for OBI. Include a brief overview paragraph and a table of contents linking to these pages:\n${topics.map(t => `- ${t.title} — ${t.description}`).join('\n')}\n\nWrite in German. Use markdown links like [Titel](Datei.md).` }
|
||||||
|
];
|
||||||
|
|
||||||
|
// Content-Pages: Nur ERSTE Seite bekommt volle Rohdaten, restliche nur Kurzfassung
|
||||||
|
for (let i = 0; i < topics.length; i++) {
|
||||||
|
const t = topics[i];
|
||||||
|
const sourceSection = i === 0
|
||||||
|
? `Quellen:\n${fullSources}`
|
||||||
|
: `Quellen (Auszug):\n${compactSources}`;
|
||||||
|
|
||||||
|
pages.push({
|
||||||
|
file: `${t.title.replace(/[^a-zA-Z0-9äöüÄÖÜß-]/g, '').replace(/\s+/g, '-')}.md`,
|
||||||
|
title: t.title,
|
||||||
|
prompt: `Create wiki/${t.title.replace(/[^a-zA-Z0-9äöüÄÖÜß-]/g, '').replace(/\s+/g, '-')}.md in German for the IFTMIN OBI wiki.\n\nTopic: ${t.title} — ${t.description}\n\n${sourceSection}\n\nWrite a well-structured markdown page with sections, tables where appropriate, and 'Siehe auch' links at the end referencing the other wiki pages.`
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
fs.mkdirSync(wikiDir, { recursive: true });
|
||||||
|
console.log(`\n${C.bold}📚 Wiki-Ingest gestartet: ${pages.length} Seiten${C.reset}\n`);
|
||||||
|
|
||||||
|
for (let i = 0; i < pages.length; i++) {
|
||||||
|
const page = pages[i];
|
||||||
|
console.log(` ${C.cyan}[${i+1}/${pages.length}]${C.reset} ${page.title} ${C.dim}(${page.file})${C.reset}`);
|
||||||
|
|
||||||
|
const history = [
|
||||||
|
{ role: 'system', content: `You are a wiki builder. Create markdown wiki pages from source material. Output the page content directly as markdown (no code block wrapper needed).` },
|
||||||
|
{ role: 'user', content: page.prompt }
|
||||||
|
];
|
||||||
|
|
||||||
|
let result = '';
|
||||||
|
let hasError = false;
|
||||||
|
try {
|
||||||
|
result = await chatCompleteFn(history, () => {}, 1500);
|
||||||
|
} catch (err) {
|
||||||
|
console.log(` ${C.red}✗${C.reset} Fehler: ${err.message}`);
|
||||||
|
hasError = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!hasError) {
|
||||||
|
result = result.trim();
|
||||||
|
// Codeblock extrahieren falls vorhanden (8B mit FILE:-Pattern, 24B-Roh-Output)
|
||||||
|
const cb = result.match(/```(?:\w+)?\n([\s\S]+?)```/);
|
||||||
|
const saveContent = cb ? cb[1].trim() : result;
|
||||||
|
fs.writeFileSync(path.join(wikiDir, page.file), saveContent, 'utf8');
|
||||||
|
console.log(` ${C.green}✓${C.reset} Gespeichert (${saveContent.length} Zeichen${cb ? '' : ', Roh-Output'})`);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Kleine Pause zwischen Requests (Server-Entlastung)
|
||||||
|
if (i < pages.length - 1) await new Promise(r => setTimeout(r, 500));
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n${C.green}✅${C.bold} Wiki-Ingest abgeschlossen${C.reset}`);
|
||||||
|
console.log(` ${C.dim}${pages.length} Seiten in ${path.relative(WORKSPACE, wikiDir)}${C.reset}`);
|
||||||
|
}
|
||||||
async function main() {
|
async function main() {
|
||||||
console.clear ? console.clear() : null;
|
console.clear ? console.clear() : null;
|
||||||
console.log(`\n${C.bold}${C.cyan}╔══════════════════════════════════════════╗${C.reset}`);
|
console.log(`\n${C.bold}${C.cyan}╔══════════════════════════════════════════╗${C.reset}`);
|
||||||
|
|
@ -501,6 +678,7 @@ async function main() {
|
||||||
:q, :quit Beenden
|
:q, :quit Beenden
|
||||||
:m Multiline-Eingabe (für Code, :send zum Senden)
|
:m Multiline-Eingabe (für Code, :send zum Senden)
|
||||||
:clear Chat-Verlauf löschen
|
:clear Chat-Verlauf löschen
|
||||||
|
:wiki Wiki-Ingest aus raw/ + AGENTS.md (Chunked-Modus)
|
||||||
|
|
||||||
${C.cyan}@datei.py${C.reset} Dateiinhalt lesen
|
${C.cyan}@datei.py${C.reset} Dateiinhalt lesen
|
||||||
${C.cyan}@verz/${C.reset} Verzeichnis auflisten
|
${C.cyan}@verz/${C.reset} Verzeichnis auflisten
|
||||||
|
|
@ -509,6 +687,13 @@ async function main() {
|
||||||
${C.cyan}@?*.ts${C.reset} Glob-Suche
|
${C.cyan}@?*.ts${C.reset} Glob-Suche
|
||||||
|
|
||||||
Das LLM kann: FILE:, DIR:, EDIT:, \`\`\`bash ausgeben
|
Das LLM kann: FILE:, DIR:, EDIT:, \`\`\`bash ausgeben
|
||||||
|
(24B: auch path:, Datei: oder Codeblock ohne Prefix)
|
||||||
|
|
||||||
|
${C.dim}Konfiguration (via ENV):
|
||||||
|
LLAMA_TIMEOUT=300000 Streaming-Timeout in ms (Default: 120000)
|
||||||
|
LLAMA_CTX_SIZE=8192 Kontext-Fenster (Default: 4096)
|
||||||
|
LLAMA_GPU_LAYERS=8 GPU-Offload-Layer (Default: 0)
|
||||||
|
LLAMA_API=url API-Endpoint (Default: http://127.0.0.1:8080/v1)${C.reset}
|
||||||
`);
|
`);
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
|
@ -517,6 +702,10 @@ async function main() {
|
||||||
console.log(` ${C.dim}Chat-Verlauf gelöscht.${C.reset}`);
|
console.log(` ${C.dim}Chat-Verlauf gelöscht.${C.reset}`);
|
||||||
continue;
|
continue;
|
||||||
}
|
}
|
||||||
|
if (trimmed === ':wiki') {
|
||||||
|
await wikiIngest(chatComplete);
|
||||||
|
continue;
|
||||||
|
}
|
||||||
if (!trimmed) continue;
|
if (!trimmed) continue;
|
||||||
|
|
||||||
// ─── Shell-Kommando-Erkennung (direkt ausführen, nicht ans LLM) ─────
|
// ─── Shell-Kommando-Erkennung (direkt ausführen, nicht ans LLM) ─────
|
||||||
|
|
@ -530,10 +719,12 @@ async function main() {
|
||||||
'file', 'type', 'printenv', 'history', 'screen', 'tmux', 'bash', 'sh',
|
'file', 'type', 'printenv', 'history', 'screen', 'tmux', 'bash', 'sh',
|
||||||
'zsh', 'source', '.',
|
'zsh', 'source', '.',
|
||||||
];
|
];
|
||||||
const firstWord = trimmed.split(/\s+/)[0];
|
// Shell-Prompt-Präfixe entfernen ($, >, #) und erstes Wort prüfen
|
||||||
const isShellCmd = SHELL_COMMANDS.includes(firstWord) && !/[?!.]$/.test(trimmed);
|
const cmdTrimmed = trimmed.replace(/^[$>#]\s*/, '');
|
||||||
|
const firstWord = cmdTrimmed.split(/\s+/)[0];
|
||||||
|
const isShellCmd = SHELL_COMMANDS.includes(firstWord) && !/[?!.]$/.test(cmdTrimmed);
|
||||||
if (isShellCmd) {
|
if (isShellCmd) {
|
||||||
const lines = trimmed.split('\n').filter(l => l.trim());
|
const lines = cmdTrimmed.split('\n').filter(l => l.trim());
|
||||||
process.stdout.write(`\n${C.dim}⚡ ${trimmed}${C.reset}\n`);
|
process.stdout.write(`\n${C.dim}⚡ ${trimmed}${C.reset}\n`);
|
||||||
for (const cmd of lines) {
|
for (const cmd of lines) {
|
||||||
try {
|
try {
|
||||||
|
|
|
||||||
1
models/LFM2-8B-A1B-UD-Q4_K_XL.gguf
Symbolic link
1
models/LFM2-8B-A1B-UD-Q4_K_XL.gguf
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
/media/daten/MiniLlama/models/LFM2-8B-A1B-UD-Q4_K_XL.gguf
|
||||||
1
models/LFM2.5-8B-A1B-Q4_K_M.gguf
Symbolic link
1
models/LFM2.5-8B-A1B-Q4_K_M.gguf
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
/media/daten/MiniLlama/models/LFM2.5-8B-A1B-Q4_K_M.gguf
|
||||||
1
models/LiquidAI_LFM2-24B-A2B-Q3_K_XL.gguf
Symbolic link
1
models/LiquidAI_LFM2-24B-A2B-Q3_K_XL.gguf
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
/media/daten/MiniLlama/models/LiquidAI_LFM2-24B-A2B-Q3_K_XL.gguf
|
||||||
1
models/LiquidAI_LFM2-8B-A1B-IQ4_XS.gguf
Symbolic link
1
models/LiquidAI_LFM2-8B-A1B-IQ4_XS.gguf
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
/media/daten/MiniLlama/models/LiquidAI_LFM2-8B-A1B-IQ4_XS.gguf
|
||||||
1
models/LiquidAI_LFM2-8B-A1B-Q3_K_XL.gguf
Symbolic link
1
models/LiquidAI_LFM2-8B-A1B-Q3_K_XL.gguf
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
/media/daten/MiniLlama/models/LiquidAI_LFM2-8B-A1B-Q3_K_XL.gguf
|
||||||
|
|
@ -1 +0,0 @@
|
||||||
/media/xray/llama/llama/models/MiniCPM5-1B-Q4_K_M.gguf
|
|
||||||
|
|
@ -1 +0,0 @@
|
||||||
/media/xray/llama/llama/models/Phi-4-mini-instruct-Q4_K_M.gguf
|
|
||||||
|
|
@ -1 +0,0 @@
|
||||||
/media/xray/llama/llama/models/Qwen3.5-4B-Q5_K_M.gguf
|
|
||||||
|
|
@ -1 +0,0 @@
|
||||||
/media/xray/llama/llama/models/gemma-3-4b-it-Q4_K_M.gguf
|
|
||||||
|
|
@ -1 +0,0 @@
|
||||||
/media/xray/llama/llama/models/gemma-4-E2B-it-Q4_K_M.gguf
|
|
||||||
|
|
@ -1 +0,0 @@
|
||||||
/media/xray/llama/llama/models/qwen2.5-1.5b-instruct-q4_k_m.gguf
|
|
||||||
|
|
@ -2,11 +2,17 @@
|
||||||
# start-server.sh — MiniLlama als API-Server starten (für CrowdCode u.a.)
|
# start-server.sh — MiniLlama als API-Server starten (für CrowdCode u.a.)
|
||||||
# Startet llama-server im Hintergrund auf http://127.0.0.1:8080
|
# Startet llama-server im Hintergrund auf http://127.0.0.1:8080
|
||||||
# Der Server läuft bis Ctrl+C oder das Terminal geschlossen wird.
|
# Der Server läuft bis Ctrl+C oder das Terminal geschlossen wird.
|
||||||
|
# Default: LFM2-8B-A1B-UD-Q4_K_XL (Testsieger)
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||||
cd "$SCRIPT_DIR"
|
cd "$SCRIPT_DIR"
|
||||||
|
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
# Default-Modell (Testsieger: beste CPU-Performance + Qualität)
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
DEFAULT_MODEL="LFM2-8B-A1B-UD-Q4_K_XL.gguf"
|
||||||
|
|
||||||
# ------------------------------------------------------------
|
# ------------------------------------------------------------
|
||||||
# Modelle aus dem models/-Ordner lesen
|
# Modelle aus dem models/-Ordner lesen
|
||||||
# ------------------------------------------------------------
|
# ------------------------------------------------------------
|
||||||
|
|
@ -29,15 +35,30 @@ echo ""
|
||||||
for i in "${!MODELS[@]}"; do
|
for i in "${!MODELS[@]}"; do
|
||||||
NAME=$(basename "${MODELS[$i]}")
|
NAME=$(basename "${MODELS[$i]}")
|
||||||
SIZE=$(numfmt --to=iec --format='%.1f' "$(stat --printf='%s' -L "${MODELS[$i]}" 2>/dev/null)" 2>/dev/null || echo "?")
|
SIZE=$(numfmt --to=iec --format='%.1f' "$(stat --printf='%s' -L "${MODELS[$i]}" 2>/dev/null)" 2>/dev/null || echo "?")
|
||||||
printf " %2d) %s (%s)\n" $((i + 1)) "$NAME" "$SIZE"
|
DEFAULT_MARKER=""
|
||||||
|
if [ "$NAME" = "$DEFAULT_MODEL" ]; then
|
||||||
|
DEFAULT_MARKER=" ← Standard"
|
||||||
|
fi
|
||||||
|
printf " %2d) %s (%s)%s\n" $((i + 1)) "$NAME" "$SIZE" "$DEFAULT_MARKER"
|
||||||
done
|
done
|
||||||
echo ""
|
echo ""
|
||||||
|
|
||||||
SELECTION=""
|
SELECTION=""
|
||||||
while [ -z "$SELECTION" ]; do
|
while [ -z "$SELECTION" ]; do
|
||||||
read -rp "Auswahl [1-$MODEL_COUNT] oder q=beenden: " INPUT
|
read -rp "Auswahl [1-$MODEL_COUNT, Enter=$DEFAULT_MODEL, q=beenden]: " INPUT
|
||||||
if [ "$INPUT" = "q" ]; then exit 0; fi
|
if [ "$INPUT" = "q" ]; then exit 0; fi
|
||||||
if [[ "$INPUT" =~ ^[0-9]+$ ]] && [ "$INPUT" -ge 1 ] && [ "$INPUT" -le "$MODEL_COUNT" ]; then
|
if [ -z "$INPUT" ]; then
|
||||||
|
for i in "${!MODELS[@]}"; do
|
||||||
|
if [[ "$(basename "${MODELS[$i]}")" == "$DEFAULT_MODEL" ]]; then
|
||||||
|
SELECTION=$i
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
if [ -z "$SELECTION" ]; then
|
||||||
|
echo "❌ Default $DEFAULT_MODEL nicht gefunden, wähle erstes."
|
||||||
|
SELECTION=0
|
||||||
|
fi
|
||||||
|
elif [[ "$INPUT" =~ ^[0-9]+$ ]] && [ "$INPUT" -ge 1 ] && [ "$INPUT" -le "$MODEL_COUNT" ]; then
|
||||||
SELECTION=$((INPUT - 1))
|
SELECTION=$((INPUT - 1))
|
||||||
else
|
else
|
||||||
echo "❌ Bitte eine Zahl zwischen 1 und $MODEL_COUNT eingeben."
|
echo "❌ Bitte eine Zahl zwischen 1 und $MODEL_COUNT eingeben."
|
||||||
|
|
|
||||||
24
start.bat
24
start.bat
|
|
@ -1,8 +1,11 @@
|
||||||
@echo off
|
@echo off
|
||||||
:: MiniLlama — Portable LLM Chat Launcher (Windows)
|
:: MiniLlama — Portable LLM Chat Launcher (Windows)
|
||||||
:: Erkennt automatisch alle .gguf-Modelle im models/-Ordner und zeigt ein Menü.
|
:: Erkennt automatisch alle .gguf-Modelle im models/-Ordner und zeigt ein Menü.
|
||||||
|
:: Default: LFM2-8B-A1B-UD-Q4_K_XL (Testsieger)
|
||||||
cd /d "%~dp0"
|
cd /d "%~dp0"
|
||||||
|
|
||||||
|
set "DEFAULT_MODEL=LFM2-8B-A1B-UD-Q4_K_XL.gguf"
|
||||||
|
|
||||||
echo.
|
echo.
|
||||||
echo =======================================
|
echo =======================================
|
||||||
echo MiniLlama — Modellauswahl
|
echo MiniLlama — Modellauswahl
|
||||||
|
|
@ -27,17 +30,30 @@ if %count% equ 0 (
|
||||||
exit /b 1
|
exit /b 1
|
||||||
)
|
)
|
||||||
|
|
||||||
:: --- Menü anzeigen ---
|
:: --- Menü anzeigen mit Default-Markierung ---
|
||||||
for /l %%i in (1,1,%count%) do (
|
for /l %%i in (1,1,%count%) do (
|
||||||
echo %%i^) !modelname[%%i]!
|
set "marker="
|
||||||
|
if "!modelname[%%i]!"=="%DEFAULT_MODEL%" set "marker= ^<-- Standard"
|
||||||
|
echo %%i^) !modelname[%%i]!!marker!
|
||||||
)
|
)
|
||||||
echo.
|
echo.
|
||||||
|
|
||||||
:: --- Auswahl ---
|
:: --- Auswahl (Default bei Enter) ---
|
||||||
:choose
|
:choose
|
||||||
set /p sel="Auswahl [1-%count%] oder q=beenden: "
|
set /p sel="Auswahl [1-%count%] oder Enter=%DEFAULT_MODEL% oder q=beenden: "
|
||||||
if /i "!sel!"=="q" exit /b 0
|
if /i "!sel!"=="q" exit /b 0
|
||||||
|
|
||||||
|
:: Leere Eingabe = Default
|
||||||
|
if "!sel!"=="" (
|
||||||
|
for /l %%i in (1,1,%count%) do (
|
||||||
|
if "!modelname[%%i]!"=="%DEFAULT_MODEL%" set idx=%%i
|
||||||
|
)
|
||||||
|
if defined idx goto launch
|
||||||
|
echo Default-Modell %DEFAULT_MODEL% nicht gefunden, nehme erstes.
|
||||||
|
set idx=1
|
||||||
|
goto launch
|
||||||
|
)
|
||||||
|
|
||||||
:: Prüfen ob Zahl
|
:: Prüfen ob Zahl
|
||||||
set "isnum="
|
set "isnum="
|
||||||
for /f "delims=0123456789" %%a in ("!sel!") do set isnum=%%a
|
for /f "delims=0123456789" %%a in ("!sel!") do set isnum=%%a
|
||||||
|
|
|
||||||
30
start.sh
30
start.sh
|
|
@ -1,11 +1,17 @@
|
||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
# MiniLlama — Portable LLM Chat Launcher (Linux)
|
# MiniLlama — Portable LLM Chat Launcher (Linux)
|
||||||
# Erkennt automatisch alle .gguf-Modelle im models/-Ordner und zeigt ein Menü.
|
# Erkennt automatisch alle .gguf-Modelle im models/-Ordner und zeigt ein Menü.
|
||||||
|
# Default: LFM2-8B-A1B-UD-Q4_K_XL (Testsieger)
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd)"
|
||||||
cd "$SCRIPT_DIR"
|
cd "$SCRIPT_DIR"
|
||||||
|
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
# Default-Modell (Testsieger: beste CPU-Performance + Qualität)
|
||||||
|
# ------------------------------------------------------------
|
||||||
|
DEFAULT_MODEL="LFM2-8B-A1B-UD-Q4_K_XL.gguf"
|
||||||
|
|
||||||
# ------------------------------------------------------------
|
# ------------------------------------------------------------
|
||||||
# Modelle aus dem models/-Ordner lesen
|
# Modelle aus dem models/-Ordner lesen
|
||||||
# ------------------------------------------------------------
|
# ------------------------------------------------------------
|
||||||
|
|
@ -29,18 +35,34 @@ echo ""
|
||||||
for i in "${!MODELS[@]}"; do
|
for i in "${!MODELS[@]}"; do
|
||||||
NAME=$(basename "${MODELS[$i]}")
|
NAME=$(basename "${MODELS[$i]}")
|
||||||
SIZE=$(numfmt --to=iec --format='%.1f' "$(stat --printf='%s' -L "${MODELS[$i]}" 2>/dev/null)" 2>/dev/null || echo "?")
|
SIZE=$(numfmt --to=iec --format='%.1f' "$(stat --printf='%s' -L "${MODELS[$i]}" 2>/dev/null)" 2>/dev/null || echo "?")
|
||||||
printf " %2d) %s (%s)\n" $((i + 1)) "$NAME" "$SIZE"
|
DEFAULT_MARKER=""
|
||||||
|
if [ "$NAME" = "$DEFAULT_MODEL" ]; then
|
||||||
|
DEFAULT_MARKER=" ← Standard"
|
||||||
|
fi
|
||||||
|
printf " %2d) %s (%s)%s\n" $((i + 1)) "$NAME" "$SIZE" "$DEFAULT_MARKER"
|
||||||
done
|
done
|
||||||
echo ""
|
echo ""
|
||||||
|
|
||||||
# ------------------------------------------------------------
|
# ------------------------------------------------------------
|
||||||
# Auswahl
|
# Auswahl (Default bei Enter)
|
||||||
# ------------------------------------------------------------
|
# ------------------------------------------------------------
|
||||||
SELECTION=""
|
SELECTION=""
|
||||||
while [ -z "$SELECTION" ]; do
|
while [ -z "$SELECTION" ]; do
|
||||||
read -rp "Auswahl [1-$MODEL_COUNT] oder q=beenden: " INPUT
|
read -rp "Auswahl [1-$MODEL_COUNT, Enter=$DEFAULT_MODEL, q=beenden]: " INPUT
|
||||||
if [ "$INPUT" = "q" ]; then exit 0; fi
|
if [ "$INPUT" = "q" ]; then exit 0; fi
|
||||||
if [[ "$INPUT" =~ ^[0-9]+$ ]] && [ "$INPUT" -ge 1 ] && [ "$INPUT" -le "$MODEL_COUNT" ]; then
|
if [ -z "$INPUT" ]; then
|
||||||
|
# Default-Modell finden
|
||||||
|
for i in "${!MODELS[@]}"; do
|
||||||
|
if [[ "$(basename "${MODELS[$i]}")" == "$DEFAULT_MODEL" ]]; then
|
||||||
|
SELECTION=$i
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
if [ -z "$SELECTION" ]; then
|
||||||
|
echo "❌ Default-Modell $DEFAULT_MODEL nicht gefunden, wähle erstes."
|
||||||
|
SELECTION=0
|
||||||
|
fi
|
||||||
|
elif [[ "$INPUT" =~ ^[0-9]+$ ]] && [ "$INPUT" -ge 1 ] && [ "$INPUT" -le "$MODEL_COUNT" ]; then
|
||||||
SELECTION=$((INPUT - 1))
|
SELECTION=$((INPUT - 1))
|
||||||
else
|
else
|
||||||
echo "❌ Bitte eine Zahl zwischen 1 und $MODEL_COUNT eingeben."
|
echo "❌ Bitte eine Zahl zwischen 1 und $MODEL_COUNT eingeben."
|
||||||
|
|
|
||||||
Loading…
Add table
Reference in a new issue