char-fork/llama.cpp

git clone https://git.t4t.associates/char-fork/llama.cpp

Marcel Petrickchore : correct typos [no ci] (#20041)92f7da00b

char/aliceai-t5-moe
brancheschar/aliceai-t5-moeall refs →
1.4 KiB41 linesraw

llama.cpp/example/sycl

This example program provides the tools for llama.cpp for SYCL on Intel GPU.

Tool

Tool NameFunctionStatus
llama-ls-sycl-deviceList all SYCL devices with ID, compute capability, max work group size, etc.Support

llama-ls-sycl-device

List all SYCL devices with ID, compute capability, max work group size, etc.

  1. Build the llama.cpp for SYCL for the specified target (using GGML_SYCL_TARGET).

  2. Enable oneAPI running environment (if GGML_SYCL_TARGET is set to INTEL -default-)

source /opt/intel/oneapi/setvars.sh
  1. Execute
./build/bin/llama-ls-sycl-device

Check the ID in startup log, like:

found 2 SYCL devices:
|  |                   |                                       |       |Max    |        |Max  |Global |                     |
|  |                   |                                       |       |compute|Max work|sub  |mem    |                     |
|ID|        Device Type|                                   Name|Version|units  |group   |group|size   |       Driver version|
|--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------|
| 0| [level_zero:gpu:0]|                Intel Arc A770 Graphics|    1.3|    512|    1024|   32| 16225M|            1.3.29138|
| 1| [level_zero:gpu:1]|                 Intel UHD Graphics 750|    1.3|     32|     512|   32| 62631M|            1.3.29138|


1# llama.cpp/example/sycl
2
3This example program provides the tools for llama.cpp for SYCL on Intel GPU.
4
5## Tool
6
7|Tool Name| Function|Status|
8|-|-|-|
9|llama-ls-sycl-device| List all SYCL devices with ID, compute capability, max work group size, etc.|Support|
10
11### llama-ls-sycl-device
12
13List all SYCL devices with ID, compute capability, max work group size, etc.
14
151. Build the llama.cpp for SYCL for the specified target *(using GGML_SYCL_TARGET)*.
16
172. Enable oneAPI running environment *(if GGML_SYCL_TARGET is set to INTEL -default-)*
18
19```
20source /opt/intel/oneapi/setvars.sh
21```
22
233. Execute
24
25```
26./build/bin/llama-ls-sycl-device
27```
28
29Check the ID in startup log, like:
30
31```
32found 2 SYCL devices:
33|  |                   |                                       |       |Max    |        |Max  |Global |                     |
34|  |                   |                                       |       |compute|Max work|sub  |mem    |                     |
35|ID|        Device Type|                                   Name|Version|units  |group   |group|size   |       Driver version|
36|--|-------------------|---------------------------------------|-------|-------|--------|-----|-------|---------------------|
37| 0| [level_zero:gpu:0]|                Intel Arc A770 Graphics|    1.3|    512|    1024|   32| 16225M|            1.3.29138|
38| 1| [level_zero:gpu:1]|                 Intel UHD Graphics 750|    1.3|     32|     512|   32| 62631M|            1.3.29138|
39
40```
41