char-fork/llama.cpp

git clone https://git.t4t.associates/char-fork/llama.cpp

Charlotte Somenc/dec in server context8300f1411

char/aliceai-t5-moe
brancheschar/aliceai-t5-moeall refs →
6 files

files

CMakeLists.txt989 B
README.md485 B
lookup-create.cpp1.2 KiB
lookup-merge.cpp1.4 KiB
lookup-stats.cpp5.7 KiB
lookup.cpp8.3 KiB

llama.cpp/examples/lookup

Demonstration of Prompt Lookup Decoding

https://github.com/apoorvumang/prompt-lookup-decoding

The key parameters for lookup decoding are ngram_min, ngram_max and n_draft. The first two determine the size of the ngrams to search for in the prompt for a match. The latter specifies how many subsequent tokens to draft if a match is found.

More info:

https://github.com/ggml-org/llama.cpp/pull/4484 https://github.com/ggml-org/llama.cpp/issues/4226