Repolode
Submit
Andyyyy64

Andyyyy64/whichllm

A CLI that detects your GPU, CPU and RAM and ranks the HuggingFace local LLMs that actually run on your hardware, by benchmarks rather than parameter count.

View on GitHub
Stars
6,767
Forks
373
Open issues
6
Last commit
yesterday

What is it?

whichllm answers the question of which local LLM you can actually run well on your machine. It detects NVIDIA, AMD, Intel, Apple Silicon and CPU-only setups, estimates VRAM needs and generation speed for each candidate, and prints a ranked list of models fetched live from the HuggingFace API.

Ranking is based on merged benchmark results such as LiveBench, Artificial Analysis, Aider, Chatbot Arena ELO and Open LLM Leaderboard, with recency-aware demotion for stale scores. Each score carries an evidence label (direct, variant, base, interpolated or self-reported), so inherited or uploader-claimed results are visibly discounted. You can simulate any GPU with --gpu before buying hardware, compare upgrade candidates, or find the GPU needed for a given model.

Beyond recommendations, the tool can download and chat with a model via llama-cpp-python, transformers or AutoAWQ/AutoGPTQ, print ready-to-run Python snippets, and emit JSON or Markdown output for scripts and issues.

Key features

  • 01Auto-detects NVIDIA, AMD, Intel, Apple Silicon and CPU-only hardware
  • 02Ranks models by merged real benchmarks with recency-aware demotion
  • 03Estimates VRAM fit, full-GPU or partial offload, and tokens per second
  • 04Simulates any GPU or multi-GPU setup with --gpu before you buy
  • 05Reverse lookup of the hardware needed for a model with plan
  • 06One-command chat via run and copy-paste Python via snippet
  • 07JSON and Markdown output for scripts and sharing

Use cases

  • Choosing a local model before buying or upgrading a GPU
  • Finding the largest model that runs smoothly on a laptop or desktop
  • Checking which GPU a specific model such as Llama 70B requires
  • Feeding the top model ID into scripts or an Ollama workflow

Quick start

  1. 1

    Run the recommendation once with no setup

    uvx whichllm@latest
  2. 2

    Simulate a GPU before buying hardware

    uvx whichllm@latest --gpu "RTX 4090"
  3. 3

    Install it for frequent use

    uv tool install whichllm

Things to know

  • Model rankings depend on live HuggingFace data and benchmark sources, so results can change over time.
  • Speed and VRAM figures are estimates, not live measurements on your machine.

Topics

#local-llm#llm#gpu#huggingface#benchmark#cli#hardware#quantization#localllm

FAQ

What is whichllm?
A CLI that detects your GPU, CPU and RAM and ranks the HuggingFace local LLMs that actually run on your hardware, by benchmarks rather than parameter count. whichllm answers the question of which local LLM you can actually run well on your machine. It detects NVIDIA, AMD, Intel, Apple Silicon and CPU-only setups, estimates VRAM needs and generation speed for each candidate, and prints a ranked list of models fetched live from the HuggingFace API.
Is whichllm free to use? What license does it have?
whichllm is open source under the MIT license. Check the license terms before using it in commercial products.
Is whichllm still maintained?
Actively maintained. The last commit to whichllm was on Oct 19, 2026.
What can I use whichllm for?
Choosing a local model before buying or upgrading a GPU Finding the largest model that runs smoothly on a laptop or desktop Checking which GPU a specific model such as Llama 70B requires Feeding the top model ID into scripts or an Ollama workflow
How do I install whichllm?
Run the recommendation once with no setup: uvx whichllm@latest Simulate a GPU before buying hardware: uvx whichllm@latest --gpu "RTX 4090" Install it for frequent use: uv tool install whichllm
Where can I get whichllm?
The source code and installation instructions are on GitHub: https://github.com/Andyyyy64/whichllm

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects