At the moment functionality in the binding.odin is enough to run simple example based on llama.cpp/examples/simple example.

1. Clone Llama.cpp and checkout to the b3735 tag:

git clone https://github.com/ggerganov/llama.cpp
cd llama.cpp
git checkout b3735

2. [Optional] Build libraries and binaries. This allows you to test build with chosen build flags.

# assume you're in the llama.cpp dir
mkdir build
cd build
cmake -DBUILD_SHARED_LIBS=ON -DGGML_CUDA=ON ..
cmake --build . --config Release --parallel 8

3. Download model you want into the models folder:

# assume you're in the project root dir
make get-gemma2b

4. [Optional] Test downloaded model with llama-cli you've built on the step (2)

./llama.cpp/build/bin/llama-cli --model models/gemma-2b.Q2_K.gguf --prompt "Once upon a time" --predict 128

5. Build and run simple example

# assume you're in the project root dir
make example-simple

This will apply Makefile.patch to the llama.cpp/Makefile, copy binding.cpp and binding.h to the llama.cpp/ folder, and build binding.a file.

Then, simple project will be compiled and linked with binding.a.

1. This is a "thin" wrapper, no custom functionality is added.
2. All command line arguments are directly passed to the llama.cpp for parsing.
3. Opaque pointers has _ptr postfix.
4. Wrapped functions has lpp_ prefix (short of llama.cpp). Those are usually form common.h.
5. Functions without lpp_ prefix are from llama.h

Here: https://github.com/ggerganov/llama.cpp/tree/b3735/

To extend current functionality, please refere to the following:

• code
  • https://github.com/ggerganov/llama.cpp/tree/b3735/common
  • https://github.com/ggerganov/llama.cpp/blob/b3735/include/llama.h
• examples
  • https://github.com/ggerganov/llama.cpp/tree/b3735/examples