char-fork/llama.cpp

git clone https://git.t4t.associates/char-fork/llama.cpp

Daniel Beveniusllama.swiftui : fix xcframework dir in README [no ci] (#12353)80a02aa85

char/aliceai-t5-moe
brancheschar/aliceai-t5-moeall refs →
1.1 KiB27 linesraw

llama.cpp/examples/llama.swiftui

Local inference of llama.cpp on an iPhone. This is a sample app that can be used as a starting point for more advanced projects.

For usage instructions and performance stats, check the following discussion: https://github.com/ggml-org/llama.cpp/discussions/4508

Building

First llama.cpp need to be built and a XCFramework needs to be created. This can be done by running the following script from the llama.cpp project root:

$ ./build-xcframework.sh

Open llama.swiftui.xcodeproj project in Xcode and you should be able to build and run the app on a simulator or a real device.

To use the framework with a different project, the XCFramework can be added to the project by adding build-apple/llama.xcframework by dragging and dropping it into the project navigator, or by manually selecting the framework in the "Frameworks, Libraries, and Embedded Content" section of the project settings.

image

Video demonstration:

https://github.com/bachittle/llama.cpp/assets/39804642/e290827a-4edb-4093-9642-2a5e399ec545

1# llama.cpp/examples/llama.swiftui
2
3Local inference of llama.cpp on an iPhone. This is a sample app that can be used as a starting
4point for more advanced projects.
5
6For usage instructions and performance stats, check the following discussion: https://github.com/ggml-org/llama.cpp/discussions/4508
7
8
9### Building
10First llama.cpp need to be built and a XCFramework needs to be created. This can be done by running
11the following script from the llama.cpp project root:
12```console
13$ ./build-xcframework.sh
14```
15Open `llama.swiftui.xcodeproj` project in Xcode and you should be able to build and run the app on
16a simulator or a real device.
17
18To use the framework with a different project, the XCFramework can be added to the project by
19adding `build-apple/llama.xcframework` by dragging and dropping it into the project navigator, or
20by manually selecting the framework in the "Frameworks, Libraries, and Embedded Content" section
21of the project settings.
22
23![image](https://github.com/ggml-org/llama.cpp/assets/1991296/2b40284f-8421-47a2-b634-74eece09a299)
24
25Video demonstration:
26
27https://github.com/bachittle/llama.cpp/assets/39804642/e290827a-4edb-4093-9642-2a5e399ec545