char-fork/llama.cpp

git clone https://git.t4t.associates/char-fork/llama.cpp

Xuan-Son Nguyenmtmd: support pocket-tts (#26871)6e62ba538

char/aliceai-t5-moe
brancheschar/aliceai-t5-moeall refs →
2.0 KiB59 linesraw

llama.cpp TTS

This is a tool to demonstrate audio generation capability in llama.cpp via libmtmd. It was added via PR #26254

Note: this tool used to serve as a demo for OuteTTS, but it was converted to a more model-agnostic tool.

Common usage

Simple usage:

llama-tts -hf ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF -p "Hello world" --output out.wav

Common params:

  • Sampling params such as --top-k, --top-p, --temp, etc.
  • -n <number_of_frames> limits the output length, e.g. -n 500. Note that how many milliseconds each frame represents varies by model
  • Core inference params such as -ngl, -b, -ub, etc.

Qwen3-TTS

Available params:

  • --tts-lang can be zh, en, de, it, pt, es, ja, ko, fr, ru (default: en)
  • --tts-speaker-file should point to a speaker reference audio file (wav, mp3)

Example usage:

llama-tts -hf ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF \
    -p "Hello world" \
    --tts-lang english \
    --tts-speaker-file speaker.mp3 \
    --output out.wav

Pocket TTS

Available params:

  • --tts-speaker-file should point to a speaker reference audio file (wav, mp3). It is required, the model produces almost no audio without it
  • Note: lang is not used, the language is a property of the weights

Example usage:

llama-tts -m pocket-tts.gguf \
    -mm mmproj-pocket-tts.gguf \
    -p "Hello world" \
    --tts-speaker-file speaker.mp3 \
    --output out.wav

Note for GGUF conversion:

The upstream repository holds one complete model per language under languages/, next to a set of shared files at the root. Convert one of the languages/<name> directories, not the root directory:

python convert_hf_to_gguf.py path/to/pocket-tts/languages/english --outfile pocket-tts.gguf
python convert_hf_to_gguf.py path/to/pocket-tts/languages/english --mmproj --outfile mmproj-pocket-tts.gguf
1# llama.cpp TTS
2
3This is a tool to demonstrate audio generation capability in llama.cpp via `libmtmd`. It was added via PR [#26254](https://github.com/ggml-org/llama.cpp/pull/26254)
4
5Note: this tool used to serve as a demo for OuteTTS, but it was converted to a more model-agnostic tool.
6
7## Common usage
8
9Simple usage:
10
11```sh
12llama-tts -hf ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF -p "Hello world" --output out.wav
13```
14
15Common params:
16- Sampling params such as `--top-k`, `--top-p`, `--temp`, etc.
17- `-n <number_of_frames>` limits the output length, e.g. `-n 500`. Note that how many milliseconds each frame represents varies by model
18- Core inference params such as `-ngl`, `-b`, `-ub`, etc.
19
20## Qwen3-TTS
21
22Available params:
23- `--tts-lang` can be `zh`, `en`, `de`, `it`, `pt`, `es`, `ja`, `ko`, `fr`, `ru` (default: `en`)
24- `--tts-speaker-file` should point to a speaker reference audio file (wav, mp3)
25
26Example usage:
27
28```sh
29llama-tts -hf ggml-org/Qwen3-TTS-12Hz-1.7B-Base-GGUF \
30    -p "Hello world" \
31    --tts-lang english \
32    --tts-speaker-file speaker.mp3 \
33    --output out.wav
34```
35
36## Pocket TTS
37
38Available params:
39- `--tts-speaker-file` should point to a speaker reference audio file (wav, mp3). It is required, the model produces almost no audio without it
40- Note: `lang` is not used, the language is a property of the weights
41
42Example usage:
43
44```sh
45llama-tts -m pocket-tts.gguf \
46    -mm mmproj-pocket-tts.gguf \
47    -p "Hello world" \
48    --tts-speaker-file speaker.mp3 \
49    --output out.wav
50```
51
52**Note for GGUF conversion:**
53
54The [upstream repository](https://huggingface.co/kyutai/pocket-tts) holds one complete model per language under `languages/`, next to a set of shared files at the root. Convert one of the `languages/<name>` directories, **not** the root directory:
55
56```sh
57python convert_hf_to_gguf.py path/to/pocket-tts/languages/english --outfile pocket-tts.gguf
58python convert_hf_to_gguf.py path/to/pocket-tts/languages/english --mmproj --outfile mmproj-pocket-tts.gguf
59```