llama.cpp

The core C/C++ engine that powers much local LLM inference.

llama.cpp is one of 20 local & on-device ai in the Tool Hut catalog. Track it alongside your other AI tools, see what it costs you each month, and resurface it when you forget you're paying for it.

• local
• inference
• open-source
• engine

Alternatives to llama.cpp

• Ollama

  Run Llama, Mistral, Gemma, and 100+ models locally with one CLI command.

• LM Studio

  Desktop GUI for downloading and running local LLMs — beginner-friendly, GPU-optimised.

• Jan

  Offline-first desktop AI app — runs local models with OpenAI-compatible API on localhost.

• AnythingLLM

  All-in-one private AI workspace — chat with documents, multi-user, local or cloud models.

• Civitai (local runner)

  AUTOMATIC1111's Stable Diffusion web UI — the standard local interface for running open image models.

• GPT4All

  Nomic's desktop app to run open LLMs privately on your machine.