char-fork/llama.cpp

git clone https://git.t4t.associates/char-fork/llama.cpp

Julien Jerphaniondocs: Adapt conda-forge package name (#26229)ad77bd31a

char/aliceai-t5-moe
1.8 KiB78 linesraw

Install pre-built version of llama.cpp

Install viaWindowsMacLinux
conda-forge✅✅✅
Winget✅
Homebrew✅✅
MacPorts✅
Nix✅✅

conda-forge (Windows, Mac and Linux)

conda-forge provides builds for:

  • CUDA (Windows and Linux)
  • Vulkan (Windows and Linux)
  • Apple Metal (macOS)
conda install -c conda-forge llama.cpp
mamba install -c conda-forge llama.cpp
# Project-local installation
pixi add llama.cpp

# Global installation
pixi global install llama.cpp

This distribution is managed on conda-forge/llama.cpp-feedstock.

Shall you have any problems, please open an issue on its issue tracker.

Winget (Windows)

winget install llama.cpp

The package is automatically updated with new llama.cpp releases. More info: https://github.com/ggml-org/llama.cpp/issues/8188

Homebrew (Mac and Linux)

brew install llama.cpp

The formula is automatically updated with new llama.cpp releases. More info: https://github.com/ggml-org/llama.cpp/discussions/7668

MacPorts (Mac)

sudo port install llama.cpp

See also: https://ports.macports.org/port/llama.cpp/details/

Nix (Mac and Linux)

nix profile install nixpkgs#llama-cpp

For flake enabled installs.

Or

nix-env --file '<nixpkgs>' --install --attr llama-cpp

For non-flake enabled installs.

This expression is automatically updated within the nixpkgs repo.

1# Install pre-built version of llama.cpp
2
3| Install via | Windows | Mac  | Linux |
4|-------------|---------|------|-------|
5| conda-forge | ✅      | ✅   | ✅   |
6| Winget      | ✅      |      |      |
7| Homebrew    |         | ✅   | ✅   |
8| MacPorts    |         | ✅   |      |
9| Nix         |         | ✅   | ✅   |
10
11## conda-forge (Windows, Mac and Linux)
12
13conda-forge provides builds for:
14 - CUDA (Windows and Linux)
15 - Vulkan (Windows and Linux)
16 - Apple Metal (macOS)
17
18```sh
19conda install -c conda-forge llama.cpp
20```
21
22```sh
23mamba install -c conda-forge llama.cpp
24```
25
26```sh
27# Project-local installation
28pixi add llama.cpp
29
30# Global installation
31pixi global install llama.cpp
32```
33
34This distribution is managed on [`conda-forge/llama.cpp-feedstock`](https://github.com/conda-forge/llama.cpp-feedstock/).
35
36Shall you have any problems, please open an issue on [its issue tracker](https://github.com/conda-forge/llama.cpp-feedstock/issues).
37
38## Winget (Windows)
39
40```sh
41winget install llama.cpp
42```
43
44The package is automatically updated with new `llama.cpp` releases. More info: https://github.com/ggml-org/llama.cpp/issues/8188
45
46## Homebrew (Mac and Linux)
47
48```sh
49brew install llama.cpp
50```
51
52The formula is automatically updated with new `llama.cpp` releases. More info: https://github.com/ggml-org/llama.cpp/discussions/7668
53
54## MacPorts (Mac)
55
56```sh
57sudo port install llama.cpp
58```
59
60See also: https://ports.macports.org/port/llama.cpp/details/
61
62## Nix (Mac and Linux)
63
64```sh
65nix profile install nixpkgs#llama-cpp
66```
67
68For flake enabled installs.
69
70Or
71
72```sh
73nix-env --file '<nixpkgs>' --install --attr llama-cpp
74```
75
76For non-flake enabled installs.
77
78This expression is automatically updated within the [nixpkgs repo](https://github.com/NixOS/nixpkgs/blob/nixos-24.05/pkgs/by-name/ll/llama-cpp/package.nix#L164).