initial: MiniLlama — portable LLM chat for USB stick (Windows + Linux)
llama.cpp b9936 binaries for x86_64 (Linux Ubuntu + Windows CPU) Dynamic model selection from models/ dir GGUF models: Gemma-4-E2B, Qwen2.5-1.5B, Qwen3.5-4B Cross-platform start.sh / start.bat scripts Deploy script for USB stick export
This commit is contained in:
commit
1cfaaf4e22
110 changed files with 283 additions and 0 deletions
64
README.md
Normal file
64
README.md
Normal file
|
|
@ -0,0 +1,64 @@
|
||||||
|
# 🦙 MiniLlama — Portable LLM Chat
|
||||||
|
|
||||||
|
**MiniLlama** ist ein portabler LLM-Chat, der direkt vom USB-Stick läuft — ohne Installation, auf **Windows und Linux**.
|
||||||
|
|
||||||
|
## Schnellstart
|
||||||
|
|
||||||
|
| OS | Befehl |
|
||||||
|
|----|--------|
|
||||||
|
| 🐧 Linux | `./start.sh` |
|
||||||
|
| 🪟 Windows | `start.bat` |
|
||||||
|
|
||||||
|
Das Skript zeigt alle Modelle aus dem `models/`-Ordner als nummeriertes Menü an. Einfach die Ziffer wählen und loschatten.
|
||||||
|
|
||||||
|
## Projektstruktur
|
||||||
|
|
||||||
|
```
|
||||||
|
MiniLlama/
|
||||||
|
├── start.sh ← Starter für Linux (Bash)
|
||||||
|
├── start.bat ← Starter für Windows (Batch)
|
||||||
|
├── bin/
|
||||||
|
│ ├── linux/ ← llama.cpp (b9936) für Linux x86_64
|
||||||
|
│ └── win/ ← llama.cpp für Windows x86_64
|
||||||
|
├── models/ ← Hier kommen die .gguf-Modelle rein
|
||||||
|
└── README.md
|
||||||
|
```
|
||||||
|
|
||||||
|
## Modelle
|
||||||
|
|
||||||
|
GGUF-Modelle (`*.gguf`) im `models/`-Ordner ablegen.
|
||||||
|
Die Start-Skripte erkennen automatisch alle vorhandenen Modelle und zeigen sie zur Auswahl an.
|
||||||
|
|
||||||
|
> **Von Quantisierungsstufen** reichen Q4_K_M und Q5_K_M für gute Qualität bei moderater Dateigröße.
|
||||||
|
|
||||||
|
## Deployment auf einen USB-Stick
|
||||||
|
|
||||||
|
1. **Modelle kopieren** (nötig für Portabilität, da Symlinks auf einem anderen Rechner brechen):
|
||||||
|
```bash
|
||||||
|
cp -r /pfad/zu/models/*.gguf /pfad/zum/stick/MiniLlama/models/
|
||||||
|
```
|
||||||
|
|
||||||
|
2. **Ganzen Ordner auf den Stick kopieren:**
|
||||||
|
```bash
|
||||||
|
# Linux → Stick
|
||||||
|
cp -r MiniLlama /media/USER/STICKNAME/
|
||||||
|
|
||||||
|
# Windows → Stick
|
||||||
|
# Einfach MiniLlama-Ordner per Explorer auf den Stick ziehen
|
||||||
|
```
|
||||||
|
|
||||||
|
3. Auf dem Zielrechner vom Stick starten:
|
||||||
|
```bash
|
||||||
|
# Linux: Terminal im Stick-Ordner öffnen
|
||||||
|
./start.sh
|
||||||
|
|
||||||
|
# Windows: Explorer öffnen, Doppelklick auf start.bat
|
||||||
|
```
|
||||||
|
|
||||||
|
## Technische Details
|
||||||
|
|
||||||
|
- **Engine:** [llama.cpp](https://github.com/ggml-org/llama.cpp) b9936
|
||||||
|
- **Plattformen:**
|
||||||
|
- Linux: x86_64 (Ubuntu-kompatibel), baut auf glibc 2.35+
|
||||||
|
- Windows: x86_64 (CPU-only, keine GPU-Treiber nötig)
|
||||||
|
- **Dateisystem:** exFAT empfohlen (Dateien > 4 GB, Windows + Linux)
|
||||||
21
bin/linux/LICENSE
Normal file
21
bin/linux/LICENSE
Normal file
|
|
@ -0,0 +1,21 @@
|
||||||
|
MIT License
|
||||||
|
|
||||||
|
Copyright (c) 2023-2026 The ggml authors
|
||||||
|
|
||||||
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
||||||
|
of this software and associated documentation files (the "Software"), to deal
|
||||||
|
in the Software without restriction, including without limitation the rights
|
||||||
|
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
||||||
|
copies of the Software, and to permit persons to whom the Software is
|
||||||
|
furnished to do so, subject to the following conditions:
|
||||||
|
|
||||||
|
The above copyright notice and this permission notice shall be included in all
|
||||||
|
copies or substantial portions of the Software.
|
||||||
|
|
||||||
|
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
||||||
|
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
||||||
|
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
||||||
|
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
||||||
|
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
||||||
|
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
||||||
|
SOFTWARE.
|
||||||
BIN
bin/linux/ggml-rpc-server
Executable file
BIN
bin/linux/ggml-rpc-server
Executable file
Binary file not shown.
1
bin/linux/libggml-base.so
Symbolic link
1
bin/linux/libggml-base.so
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
libggml-base.so.0
|
||||||
1
bin/linux/libggml-base.so.0
Symbolic link
1
bin/linux/libggml-base.so.0
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
libggml-base.so.0.15.3
|
||||||
BIN
bin/linux/libggml-base.so.0.15.3
Executable file
BIN
bin/linux/libggml-base.so.0.15.3
Executable file
Binary file not shown.
BIN
bin/linux/libggml-cpu-alderlake.so
Executable file
BIN
bin/linux/libggml-cpu-alderlake.so
Executable file
Binary file not shown.
BIN
bin/linux/libggml-cpu-cannonlake.so
Executable file
BIN
bin/linux/libggml-cpu-cannonlake.so
Executable file
Binary file not shown.
BIN
bin/linux/libggml-cpu-cascadelake.so
Executable file
BIN
bin/linux/libggml-cpu-cascadelake.so
Executable file
Binary file not shown.
BIN
bin/linux/libggml-cpu-cooperlake.so
Executable file
BIN
bin/linux/libggml-cpu-cooperlake.so
Executable file
Binary file not shown.
BIN
bin/linux/libggml-cpu-haswell.so
Executable file
BIN
bin/linux/libggml-cpu-haswell.so
Executable file
Binary file not shown.
BIN
bin/linux/libggml-cpu-icelake.so
Executable file
BIN
bin/linux/libggml-cpu-icelake.so
Executable file
Binary file not shown.
BIN
bin/linux/libggml-cpu-ivybridge.so
Executable file
BIN
bin/linux/libggml-cpu-ivybridge.so
Executable file
Binary file not shown.
BIN
bin/linux/libggml-cpu-piledriver.so
Executable file
BIN
bin/linux/libggml-cpu-piledriver.so
Executable file
Binary file not shown.
BIN
bin/linux/libggml-cpu-sandybridge.so
Executable file
BIN
bin/linux/libggml-cpu-sandybridge.so
Executable file
Binary file not shown.
BIN
bin/linux/libggml-cpu-sapphirerapids.so
Executable file
BIN
bin/linux/libggml-cpu-sapphirerapids.so
Executable file
Binary file not shown.
BIN
bin/linux/libggml-cpu-skylakex.so
Executable file
BIN
bin/linux/libggml-cpu-skylakex.so
Executable file
Binary file not shown.
BIN
bin/linux/libggml-cpu-sse42.so
Executable file
BIN
bin/linux/libggml-cpu-sse42.so
Executable file
Binary file not shown.
BIN
bin/linux/libggml-cpu-x64.so
Executable file
BIN
bin/linux/libggml-cpu-x64.so
Executable file
Binary file not shown.
BIN
bin/linux/libggml-cpu-zen4.so
Executable file
BIN
bin/linux/libggml-cpu-zen4.so
Executable file
Binary file not shown.
BIN
bin/linux/libggml-rpc.so
Executable file
BIN
bin/linux/libggml-rpc.so
Executable file
Binary file not shown.
1
bin/linux/libggml.so
Symbolic link
1
bin/linux/libggml.so
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
libggml.so.0
|
||||||
1
bin/linux/libggml.so.0
Symbolic link
1
bin/linux/libggml.so.0
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
libggml.so.0.15.3
|
||||||
BIN
bin/linux/libggml.so.0.15.3
Executable file
BIN
bin/linux/libggml.so.0.15.3
Executable file
Binary file not shown.
BIN
bin/linux/libllama-batched-bench-impl.so
Executable file
BIN
bin/linux/libllama-batched-bench-impl.so
Executable file
Binary file not shown.
BIN
bin/linux/libllama-bench-impl.so
Executable file
BIN
bin/linux/libllama-bench-impl.so
Executable file
Binary file not shown.
BIN
bin/linux/libllama-cli-impl.so
Executable file
BIN
bin/linux/libllama-cli-impl.so
Executable file
Binary file not shown.
1
bin/linux/libllama-common.so
Symbolic link
1
bin/linux/libllama-common.so
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
libllama-common.so.0
|
||||||
1
bin/linux/libllama-common.so.0
Symbolic link
1
bin/linux/libllama-common.so.0
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
libllama-common.so.0.0.9936
|
||||||
BIN
bin/linux/libllama-common.so.0.0.9936
Executable file
BIN
bin/linux/libllama-common.so.0.0.9936
Executable file
Binary file not shown.
BIN
bin/linux/libllama-completion-impl.so
Executable file
BIN
bin/linux/libllama-completion-impl.so
Executable file
Binary file not shown.
BIN
bin/linux/libllama-fit-params-impl.so
Executable file
BIN
bin/linux/libllama-fit-params-impl.so
Executable file
Binary file not shown.
BIN
bin/linux/libllama-perplexity-impl.so
Executable file
BIN
bin/linux/libllama-perplexity-impl.so
Executable file
Binary file not shown.
BIN
bin/linux/libllama-quantize-impl.so
Executable file
BIN
bin/linux/libllama-quantize-impl.so
Executable file
Binary file not shown.
BIN
bin/linux/libllama-server-impl.so
Executable file
BIN
bin/linux/libllama-server-impl.so
Executable file
Binary file not shown.
1
bin/linux/libllama.so
Symbolic link
1
bin/linux/libllama.so
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
libllama.so.0
|
||||||
1
bin/linux/libllama.so.0
Symbolic link
1
bin/linux/libllama.so.0
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
libllama.so.0.0.9936
|
||||||
BIN
bin/linux/libllama.so.0.0.9936
Executable file
BIN
bin/linux/libllama.so.0.0.9936
Executable file
Binary file not shown.
1
bin/linux/libmtmd.so
Symbolic link
1
bin/linux/libmtmd.so
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
libmtmd.so.0
|
||||||
1
bin/linux/libmtmd.so.0
Symbolic link
1
bin/linux/libmtmd.so.0
Symbolic link
|
|
@ -0,0 +1 @@
|
||||||
|
libmtmd.so.0.0.9936
|
||||||
BIN
bin/linux/libmtmd.so.0.0.9936
Executable file
BIN
bin/linux/libmtmd.so.0.0.9936
Executable file
Binary file not shown.
BIN
bin/linux/llama
Executable file
BIN
bin/linux/llama
Executable file
Binary file not shown.
BIN
bin/linux/llama-batched-bench
Executable file
BIN
bin/linux/llama-batched-bench
Executable file
Binary file not shown.
BIN
bin/linux/llama-bench
Executable file
BIN
bin/linux/llama-bench
Executable file
Binary file not shown.
BIN
bin/linux/llama-cli
Executable file
BIN
bin/linux/llama-cli
Executable file
Binary file not shown.
BIN
bin/linux/llama-completion
Executable file
BIN
bin/linux/llama-completion
Executable file
Binary file not shown.
BIN
bin/linux/llama-debug-template-parser
Executable file
BIN
bin/linux/llama-debug-template-parser
Executable file
Binary file not shown.
BIN
bin/linux/llama-fit-params
Executable file
BIN
bin/linux/llama-fit-params
Executable file
Binary file not shown.
BIN
bin/linux/llama-gemma3-cli
Executable file
BIN
bin/linux/llama-gemma3-cli
Executable file
Binary file not shown.
BIN
bin/linux/llama-gguf-split
Executable file
BIN
bin/linux/llama-gguf-split
Executable file
Binary file not shown.
BIN
bin/linux/llama-imatrix
Executable file
BIN
bin/linux/llama-imatrix
Executable file
Binary file not shown.
BIN
bin/linux/llama-llava-cli
Executable file
BIN
bin/linux/llama-llava-cli
Executable file
Binary file not shown.
BIN
bin/linux/llama-minicpmv-cli
Executable file
BIN
bin/linux/llama-minicpmv-cli
Executable file
Binary file not shown.
BIN
bin/linux/llama-mtmd-cli
Executable file
BIN
bin/linux/llama-mtmd-cli
Executable file
Binary file not shown.
BIN
bin/linux/llama-mtmd-debug
Executable file
BIN
bin/linux/llama-mtmd-debug
Executable file
Binary file not shown.
BIN
bin/linux/llama-perplexity
Executable file
BIN
bin/linux/llama-perplexity
Executable file
Binary file not shown.
BIN
bin/linux/llama-quantize
Executable file
BIN
bin/linux/llama-quantize
Executable file
Binary file not shown.
BIN
bin/linux/llama-qwen2vl-cli
Executable file
BIN
bin/linux/llama-qwen2vl-cli
Executable file
Binary file not shown.
BIN
bin/linux/llama-results
Executable file
BIN
bin/linux/llama-results
Executable file
Binary file not shown.
BIN
bin/linux/llama-server
Executable file
BIN
bin/linux/llama-server
Executable file
Binary file not shown.
BIN
bin/linux/llama-template-analysis
Executable file
BIN
bin/linux/llama-template-analysis
Executable file
Binary file not shown.
BIN
bin/linux/llama-tokenize
Executable file
BIN
bin/linux/llama-tokenize
Executable file
Binary file not shown.
BIN
bin/linux/llama-tts
Executable file
BIN
bin/linux/llama-tts
Executable file
Binary file not shown.
BIN
bin/win/ggml-base.dll
Executable file
BIN
bin/win/ggml-base.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml-cpu-alderlake.dll
Executable file
BIN
bin/win/ggml-cpu-alderlake.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml-cpu-cannonlake.dll
Executable file
BIN
bin/win/ggml-cpu-cannonlake.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml-cpu-cascadelake.dll
Executable file
BIN
bin/win/ggml-cpu-cascadelake.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml-cpu-cooperlake.dll
Executable file
BIN
bin/win/ggml-cpu-cooperlake.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml-cpu-haswell.dll
Executable file
BIN
bin/win/ggml-cpu-haswell.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml-cpu-icelake.dll
Executable file
BIN
bin/win/ggml-cpu-icelake.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml-cpu-ivybridge.dll
Executable file
BIN
bin/win/ggml-cpu-ivybridge.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml-cpu-piledriver.dll
Executable file
BIN
bin/win/ggml-cpu-piledriver.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml-cpu-sandybridge.dll
Executable file
BIN
bin/win/ggml-cpu-sandybridge.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml-cpu-sapphirerapids.dll
Executable file
BIN
bin/win/ggml-cpu-sapphirerapids.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml-cpu-skylakex.dll
Executable file
BIN
bin/win/ggml-cpu-skylakex.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml-cpu-sse42.dll
Executable file
BIN
bin/win/ggml-cpu-sse42.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml-cpu-x64.dll
Executable file
BIN
bin/win/ggml-cpu-x64.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml-cpu-zen4.dll
Executable file
BIN
bin/win/ggml-cpu-zen4.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml-rpc.dll
Executable file
BIN
bin/win/ggml-rpc.dll
Executable file
Binary file not shown.
BIN
bin/win/ggml.dll
Executable file
BIN
bin/win/ggml.dll
Executable file
Binary file not shown.
BIN
bin/win/libomp140.x86_64.dll
Executable file
BIN
bin/win/libomp140.x86_64.dll
Executable file
Binary file not shown.
BIN
bin/win/llama-batched-bench.exe
Executable file
BIN
bin/win/llama-batched-bench.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-bench.exe
Executable file
BIN
bin/win/llama-bench.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-cli.exe
Executable file
BIN
bin/win/llama-cli.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-completion.exe
Executable file
BIN
bin/win/llama-completion.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-fit-params.exe
Executable file
BIN
bin/win/llama-fit-params.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-gemma3-cli.exe
Executable file
BIN
bin/win/llama-gemma3-cli.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-gguf-split.exe
Executable file
BIN
bin/win/llama-gguf-split.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-imatrix.exe
Executable file
BIN
bin/win/llama-imatrix.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-llava-cli.exe
Executable file
BIN
bin/win/llama-llava-cli.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-minicpmv-cli.exe
Executable file
BIN
bin/win/llama-minicpmv-cli.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-mtmd-cli.exe
Executable file
BIN
bin/win/llama-mtmd-cli.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-mtmd-debug.exe
Executable file
BIN
bin/win/llama-mtmd-debug.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-perplexity.exe
Executable file
BIN
bin/win/llama-perplexity.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-quantize.exe
Executable file
BIN
bin/win/llama-quantize.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-qwen2vl-cli.exe
Executable file
BIN
bin/win/llama-qwen2vl-cli.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-results.exe
Executable file
BIN
bin/win/llama-results.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-server.exe
Executable file
BIN
bin/win/llama-server.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-template-analysis.exe
Executable file
BIN
bin/win/llama-template-analysis.exe
Executable file
Binary file not shown.
BIN
bin/win/llama-tokenize.exe
Executable file
BIN
bin/win/llama-tokenize.exe
Executable file
Binary file not shown.
Some files were not shown because too many files have changed in this diff Show more
Loading…
Add table
Reference in a new issue