Repolode
Submit
hr98w

hr98w/jev-visual

A small educational project for running vision-language model inference on Apple Silicon with MLX, scoring candidate answers about one image.

View on GitHub
Stars
320
Forks
29
Open issues
1
Last commit
28 days ago

What is it?

Jev Visual is a runnable experiment in vision-language model inference on Apple Silicon Macs. It uses Qwen3.5-0.8B through MLX to answer several questions about one image at once, such as choosing an option, judging yes or no, or scoring ordered levels. Instead of generating structured output token by token, it reuses a shared multimodal context and reads scores for candidate answers directly from model logits.

The project ships a local browser UI, a command-line tool, and an HTTP API. The inference path is split into readable modules for image preprocessing, adapter-level prefill and cache reuse, candidate scoring, and result assembly, so developers can follow how the pipeline works. It uses existing model weights with no training or calibration, and only the Qwen3.5 adapter is verified. Candidate probabilities are relative to the options supplied, not estimates of correctness.

The repository also includes benchmarks comparing generated JSON, independent candidate scoring, and shared-prefix batched scoring, along with visual game demos built on the same approach.

Key features

  • 01Runs Qwen3.5-0.8B locally on Apple Silicon using MLX.
  • 02Answers multiple questions about one image with shared context reuse.
  • 03Scores candidate answers from model logits instead of generating JSON.
  • 04Provides a local browser UI, CLI, and HTTP API.
  • 05Includes benchmarks and scoring verification scripts.

Use cases

  • Learning how vision-language model inference and candidate scoring work.
  • Prototyping image classification or multiple-choice questions on a Mac without cloud APIs.
  • Benchmarking generated output against direct logit scoring for structured answers.
  • Experimenting with visual demos such as gesture control or sorting classification.

Quick start

  1. 1

    Create a virtual environment with Python 3.13

    uv venv --python 3.13
  2. 2

    Activate the environment and install dependencies

    source .venv/bin/activate && uv pip install -r requirements-lock.txt && uv pip install --no-deps -e .
  3. 3

    Download the model weights

    jev-visual-download
  4. 4

    Start the local server, then open http://127.0.0.1:8788 in a browser

    JEV_VISUAL_MODEL_PATH=.models/Qwen3.5-0.8B-4bit uvicorn jev_visual.server:app --host 127.0.0.1 --port 8788

Things to know

  • Requires an Apple Silicon Mac with Metal; it does not run on other platforms.
  • First setup downloads roughly 596 MiB of model weights.
  • Only the Qwen3.5 adapter is verified, and the server is intended for local use only.

Topics

#vision-language-model#mlx#apple-silicon#inference#candidate-scoring#multimodal#qwen#python

FAQ

What is jev-visual?
A small educational project for running vision-language model inference on Apple Silicon with MLX, scoring candidate answers about one image. Jev Visual is a runnable experiment in vision-language model inference on Apple Silicon Macs. It uses Qwen3.5-0.8B through MLX to answer several questions about one image at once, such as choosing an option, judging yes or no, or scoring ordered levels. Instead of generating structured output token by token, it reuses a shared multimodal context and reads scores for candidate answers directly from model logits.
Is jev-visual free to use? What license does it have?
jev-visual is open source under the MIT license. Check the license terms before using it in commercial products.
Is jev-visual still maintained?
Actively maintained. The last commit to jev-visual was on Sep 21, 2026.
What can I use jev-visual for?
Learning how vision-language model inference and candidate scoring work. Prototyping image classification or multiple-choice questions on a Mac without cloud APIs. Benchmarking generated output against direct logit scoring for structured answers. Experimenting with visual demos such as gesture control or sorting classification.
How do I install jev-visual?
Create a virtual environment with Python 3.13: uv venv --python 3.13 Activate the environment and install dependencies: source .venv/bin/activate && uv pip install -r requirements-lock.txt && uv pip install --no-deps -e . Download the model weights: jev-visual-download Start the local server, then open http://127.0.0.1:8788 in a browser: JEV_VISUAL_MODEL_PATH=.models/Qwen3.5-0.8B-4bit uvicorn jev_visual.server:app --host 127.0.0.1 --port 8788
Where can I get jev-visual?
The source code and installation instructions are on GitHub: https://github.com/hr98w/jev-visual

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects