char-fork/llama.cpp

git clone https://git.t4t.associates/char-fork/llama.cpp

Daniel Beveniusexamples : add missing code block end marker [no ci] (#17756)817d743cc

char/aliceai-t5-moe
1.2 KiB35 linesraw

llama.cpp/example/simple-cmake-pkg

This program builds simple using a relocatable CMake package. It serves as an example of using the find_package() CMake command to conveniently include llama.cpp in projects which live outside of the source tree.

Building

Because this example is "outside of the source tree", it is important to first build/install llama.cpp using CMake. An example is provided here, but please see the llama.cpp build instructions for more detailed build instructions.

Considerations

When hardware acceleration libraries are used (e.g. CUDA, Metal, Vulkan, etc.), the appropriate dependencies will be searched for automatically. So, for example, when finding a package

Build llama.cpp and install to llama.cpp/inst

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -S . -B build
cmake --build build
cmake --install build --prefix inst

Build simple-cmake-pkg

cd examples/simple-cmake-pkg
cmake -S . -B build -DCMAKE_PREFIX_PATH=../../inst/lib/cmake
cmake --build build

Run simple-cmake-pkg

./build/llama-simple-cmake-pkg -m ./models/llama-7b-v2/ggml-model-f16.gguf "Hello my name is"
1# llama.cpp/example/simple-cmake-pkg
2
3This program builds [simple](../simple) using a relocatable CMake package. It serves as an example of using the `find_package()` CMake command to conveniently include [llama.cpp](https://github.com/ggml-org/llama.cpp) in projects which live outside of the source tree.
4
5## Building
6
7Because this example is "outside of the source tree", it is important to first build/install llama.cpp using CMake. An example is provided here, but please see the [llama.cpp build instructions](../..) for more detailed build instructions.
8
9### Considerations
10
11When hardware acceleration libraries are used (e.g. CUDA, Metal, Vulkan, etc.), the appropriate dependencies will be searched for automatically. So, for example, when finding a package
12
13### Build llama.cpp and install to llama.cpp/inst
14
15```sh
16git clone https://github.com/ggml-org/llama.cpp
17cd llama.cpp
18cmake -S . -B build
19cmake --build build
20cmake --install build --prefix inst
21```
22
23### Build simple-cmake-pkg
24
25```sh
26cd examples/simple-cmake-pkg
27cmake -S . -B build -DCMAKE_PREFIX_PATH=../../inst/lib/cmake
28cmake --build build
29```
30
31### Run simple-cmake-pkg
32
33```sh
34./build/llama-simple-cmake-pkg -m ./models/llama-7b-v2/ggml-model-f16.gguf "Hello my name is"
35```