char-fork/llama.cpp

git clone https://git.t4t.associates/char-fork/llama.cpp

Georgi Gerganovmake : deprecate (#10514)8648c5210

char/aliceai-t5-moe
brancheschar/aliceai-t5-moeall refs →
1.6 KiB60 linesraw

BLIS Installation Manual

BLIS is a portable software framework for high-performance BLAS-like dense linear algebra libraries. It has received awards and recognition, including the 2023 James H. Wilkinson Prize for Numerical Software and the 2020 SIAM Activity Group on Supercomputing Best Paper Prize. BLIS provides a new BLAS-like API and a compatibility layer for traditional BLAS routine calls. It offers features such as object-based API, typed API, BLAS and CBLAS compatibility layers.

Project URL: https://github.com/flame/blis

Prepare:

Compile BLIS:

git clone https://github.com/flame/blis
cd blis
./configure --enable-cblas -t openmp,pthreads auto
# will install to /usr/local/ by default.
make -j

Install BLIS:

sudo make install

We recommend using openmp since it's easier to modify the cores being used.

llama.cpp compilation

CMake:

mkdir build
cd build
cmake -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=FLAME ..
make -j

llama.cpp execution

According to the BLIS documentation, we could set the following environment variables to modify the behavior of openmp:

export GOMP_CPU_AFFINITY="0-19"
export BLIS_NUM_THREADS=14

And then run the binaries as normal.

Intel specific issue

Some might get the error message saying that libimf.so cannot be found. Please follow this stackoverflow page.

Reference:

  1. https://github.com/flame/blis#getting-started
  2. https://github.com/flame/blis/blob/master/docs/Multithreading.md
1BLIS Installation Manual
2------------------------
3
4BLIS is a portable software framework for high-performance BLAS-like dense linear algebra libraries. It has received awards and recognition, including the 2023 James H. Wilkinson Prize for Numerical Software and the 2020 SIAM Activity Group on Supercomputing Best Paper Prize. BLIS provides a new BLAS-like API and a compatibility layer for traditional BLAS routine calls. It offers features such as object-based API, typed API, BLAS and CBLAS compatibility layers.
5
6Project URL: https://github.com/flame/blis
7
8### Prepare:
9
10Compile BLIS:
11
12```bash
13git clone https://github.com/flame/blis
14cd blis
15./configure --enable-cblas -t openmp,pthreads auto
16# will install to /usr/local/ by default.
17make -j
18```
19
20Install BLIS:
21
22```bash
23sudo make install
24```
25
26We recommend using openmp since it's easier to modify the cores being used.
27
28### llama.cpp compilation
29
30CMake:
31
32```bash
33mkdir build
34cd build
35cmake -DGGML_BLAS=ON -DGGML_BLAS_VENDOR=FLAME ..
36make -j
37```
38
39### llama.cpp execution
40
41According to the BLIS documentation, we could set the following
42environment variables to modify the behavior of openmp:
43
44```bash
45export GOMP_CPU_AFFINITY="0-19"
46export BLIS_NUM_THREADS=14
47```
48
49And then run the binaries as normal.
50
51
52### Intel specific issue
53
54Some might get the error message saying that `libimf.so` cannot be found.
55Please follow this [stackoverflow page](https://stackoverflow.com/questions/70687930/intel-oneapi-2022-libimf-so-no-such-file-or-directory-during-openmpi-compila).
56
57### Reference:
58
591. https://github.com/flame/blis#getting-started
602. https://github.com/flame/blis/blob/master/docs/Multithreading.md