Llamma.cpp

If you run LLMs locally on a laptop, phone, or Raspberry Pi, you're probably already running llama.cpp — you just didn't know it.

ggml-org/llama.cpp is LLM inference in C/C++ — no Python stack, no cloud required. Created in March 2023 and now with 127K+ stars and 22K+ forks on GitHub, it's the most popular way to run open models locally. MIT licensed, blazing fast, and built on the ggml tensor library.

Why it won:

• Portable & tiny: Pure C/C++ with CPU-first design + acceleration for Metal, CUDA, Vulkan, and OpenCL. Runs on Mac, Windows, Linux, even mobile.
• Quantization + GGUF: Made 4-bit quantization and the GGUF format standard — so a 27B model actually fits and runs at ~4 tok/s on CPU alone.
• Complete toolkit: llama-server (OpenAI-compatible API), embeddings, tokenization, and bindings for everything from Python to Node.
• Private by default: Your weights, your prompts, your machine. No API key.

That's why it's become invisible infrastructure. It powers the backend for PI, OpenClaw, and OpenCode — giving those agents fast, local, private inference without calling a hosted API.

If you want to understand modern local AI, start here: git clone https://github.com/ggml-org/llama.cpp.git — and run a model that never leaves your device.

Repo: https://github.com/ggml-org/llama.cpp | C++ | MIT | homepage: llama.app