char-fork/llama.cpp

git clone https://git.t4t.associates/char-fork/llama.cpp

Francisco Herreragguf-split : clarify operation of gguf-split (#19749)8fc17493c

char/aliceai-t5-moe
505 B10 linesraw

GGUF split Example

CLI to split / merge GGUF files.

Command line options:

  • --split: split GGUF to multiple GGUF, default operation.
  • --split-max-size: max size per split in M or G, f.ex. 500M or 2G.
  • --split-max-tensors: maximum tensors in each split: default(128)
  • --merge: merge multiple GGUF to a single GGUF. You only need to specify the name of the first GGUF to merge, the name of the merged GGUF, and the CLI will find the other GGUFs it needs within the same folder.
1## GGUF split Example
2
3CLI to split / merge GGUF files.
4
5**Command line options:**
6
7- `--split`: split GGUF to multiple GGUF, default operation.
8- `--split-max-size`: max size per split in `M` or `G`, f.ex. `500M` or `2G`.
9- `--split-max-tensors`: maximum tensors in each split: default(128)
10- `--merge`: merge multiple GGUF to a single GGUF. You only need to specify the name of the first GGUF to merge, the name of the merged GGUF, and the CLI will find the other GGUFs it needs within the same folder.