Repolode
Submit
shanselman

shanselman/gputaskman

A terminal UI for Windows and NVIDIA that shows per-process GPU memory, engine activity, and the LLM weights and KV-cache inside each GPU workload.

View on GitHub
Stars
435
Forks
20
Open issues
1
Last commit
14 hours ago

What is it?

gputaskman is an htop-style monitor for NVIDIA GPUs on Windows. Task Manager shows how much GPU memory is in use but not what is using it. This tool combines Windows GPU counters with NVML device readings to break usage down by process, showing local GPU memory, GPU-accessible system RAM, engine activity, and detected runtimes such as CUDA, Vulkan, Direct3D, ggml and ONNX Runtime.

For local AI workloads, it identifies candidate models from mapped files, command-line arguments, and GGUF headers, then estimates weight and KV-cache sizes. Where llama.cpp buffer logs or local Ollama and LM Studio APIs are available, it adds allocated buffer sizes and live model state. Identification is labeled by evidence strength, and the project notes that a loaded DLL or mapped file does not by itself prove GPU placement.

The interface is a live terminal dashboard with device charts and a sortable process table, plus one-shot text and JSON output for scripting. Portable builds for x64 and ARM64 are published on the releases page, and the project can also run from source with uv or pip.

Key features

  • 01Shows per-process local GPU memory, system-RAM usage and engine activity.
  • 02Identifies loaded models from mapped files, launch arguments and GGUF headers.
  • 03Estimates llama.cpp weight and KV-cache sizes, using buffer logs when available.
  • 04Reads model state from local Ollama and LM Studio APIs when they are running.
  • 05Offers a live TUI as well as one-shot text and JSON output.

Use cases

  • Finding out why a local LLM or other AI workload is using most of the GPU memory on a Windows machine.
  • Spotting GPU-accessible system RAM use that may mean slower fallback inference.
  • Checking which local model and context size a llama-server or Ollama process is holding.
  • Capturing GPU usage snapshots in JSON for scripts or bug reports.

Quick start

  1. 1

    Install Git for Windows and uv (e.g. with WinGet)

    winget install --id Git.Git --exact
  2. 2

    Install uv

    winget install --id astral-sh.uv --exact
  3. 3

    Clone the repository

    git clone https://github.com/shanselman/gputaskman.git
  4. 4

    Run the TUI from the project folder with uv

    cd gputaskman uv run --locked gputaskman

Things to know

  • Windows and NVIDIA only; CPU-only, NPU-only and cloud workloads are out of scope.
  • Requires an NVIDIA GPU with a current Windows WDDM driver.
  • Early, experimental software; classifications and GGUF-based breakdowns are estimates.
  • Multi-GPU attribution is not yet reliable; single-NVIDIA-GPU is the validated configuration.

In collections

Terminal UI appsRun LLMs locallyOpenTelemetry & observability platforms

Topics

#gpu-monitoring#nvidia#windows#llm#vram#terminal-ui#llama-cpp#observability

FAQ

What is gputaskman?
A terminal UI for Windows and NVIDIA that shows per-process GPU memory, engine activity, and the LLM weights and KV-cache inside each GPU workload. gputaskman is an htop-style monitor for NVIDIA GPUs on Windows. Task Manager shows how much GPU memory is in use but not what is using it. This tool combines Windows GPU counters with NVML device readings to break usage down by process, showing local GPU memory, GPU-accessible system RAM, engine activity, and detected runtimes such as CUDA, Vulkan, Direct3D, ggml and ONNX Runtime.
Is gputaskman free to use? What license does it have?
gputaskman is open source under the MIT license. Check the license terms before using it in commercial products.
Is gputaskman still maintained?
Actively maintained. The last commit to gputaskman was on Nov 7, 2026.
What can I use gputaskman for?
Finding out why a local LLM or other AI workload is using most of the GPU memory on a Windows machine. Spotting GPU-accessible system RAM use that may mean slower fallback inference. Checking which local model and context size a llama-server or Ollama process is holding. Capturing GPU usage snapshots in JSON for scripts or bug reports.
How do I install gputaskman?
Install Git for Windows and uv (e.g. with WinGet): winget install --id Git.Git --exact Install uv: winget install --id astral-sh.uv --exact Clone the repository: git clone https://github.com/shanselman/gputaskman.git Run the TUI from the project folder with uv: cd gputaskman uv run --locked gputaskman
Where can I get gputaskman?
The source code and installation instructions are on GitHub: https://github.com/shanselman/gputaskman

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects