Repolode
Submit
RLinf

RLinf/APXinf-robo

Python bindings and robot-policy layer on the ApxInf Rust inference engine, serving VLA models like PI-0.5 on Jetson Thor/Orin and RTX GPUs.

View on GitHub
Stars
227
Forks
28
Open issues
1
Last commit
21 days ago

What is it?

APXinf-robo connects the ApxInf edge inference engine to embodied AI workloads. It provides a Python API and an OpenPI-compatible websocket server for running vision-language-action (VLA) models such as PI-0.5, PI0-FAST, GR00T N1.7 and Qwen-Drive on NVIDIA Jetson AGX Thor and Orin, and on RTX 4090 GPUs.

The package offers three entry points: a bare model interface that takes pre-resized frames and returns normalized actions, a robot-policy interface that handles resizing, tokenization, normalization and the flow sampler from raw observations, and a websocket server. Robot presets such as franka_libero and unitree_g1 define camera fields, state encoding and action width. Supported precisions are BF16, FP8 (Thor only) and INT8, with FP8 requiring per-tensor calibration data.

The repository also includes benchmark scripts for latency at several layers and a LIBERO-10 evaluation harness, and it vendors CUDA kernels, CUTLASS and FlashAttention sources. The engine itself is a git submodule at apxinf/, and the build compiles kernels for the GPU architecture of the machine it runs on.

Key features

  • 01Runs PI-0.5, PI0-FAST, GR00T N1.7 and Qwen-Drive models on Jetson and RTX GPUs.
  • 02Exposes a Python API at bare-model, policy and websocket server levels.
  • 03Serves an OpenPI-compatible websocket protocol so existing openpi-client code can connect.
  • 04Supports BF16, INT8 and Thor-only FP8 precisions.
  • 05Includes latency benchmarks and LIBERO-10 task evaluation scripts.

Use cases

  • Deploying a VLA policy on a Jetson robot with low-latency inference.
  • Replacing an OpenPI server with a faster engine without changing the client code.
  • Benchmarking VLA model latency across precisions on Thor, Orin or RTX 4090.
  • Evaluating a π0.5 checkpoint on the LIBERO-10 simulation suite.

Quick start

  1. 1

    Clone the repo with submodules

    git clone --recursive https://github.com/RLinf/APXinf-robo.git
  2. 2

    Create a virtual environment and install maturin

    python3 -m venv .venv && source .venv/bin/activate && pip install maturin
  3. 3

    Build and install the ApxInf Python binding and Robo package

    CARGO_TARGET_DIR=target/wheel maturin build --release --features cuda --auditwheel skip -m apxinf/crates/apxinf-py/Cargo.toml && pip install --force-reinstall target/wheel/wheels/apxinf_py-*.whl && pip install -e "./apxinf/python/apxinf[serving]" --config-settings editable_mode=strict && pip install -e ".[libero,serve]"
  4. 4

    Run the PI-0.5 benchmark with random weights

    python scripts/bench_pi05.py --random-weights --precision bf16 --layer l1 --samples 5

Things to know

  • Requires an NVIDIA GPU, CUDA toolkit, Rust and cmake, and kernels are compiled for the specific deployment architecture.
  • FP8 precision works only on Jetson AGX Thor, since Orin lacks FP8 Tensor Cores.
  • Running a real model requires a checkpoint directory that is not included with the package.

Topics

#vla#robotics#embodied-ai#inference-engine#cuda#jetson#tensorrt-alternative#python

FAQ

What is APXinf-robo?
Python bindings and robot-policy layer on the ApxInf Rust inference engine, serving VLA models like PI-0.5 on Jetson Thor/Orin and RTX GPUs. APXinf-robo connects the ApxInf edge inference engine to embodied AI workloads. It provides a Python API and an OpenPI-compatible websocket server for running vision-language-action (VLA) models such as PI-0.5, PI0-FAST, GR00T N1.7 and Qwen-Drive on NVIDIA Jetson AGX Thor and Orin, and on RTX 4090 GPUs.
Is APXinf-robo free to use? What license does it have?
APXinf-robo is open source under the Apache-2.0 license. Check the license terms before using it in commercial products.
Is APXinf-robo still maintained?
Actively maintained. The last commit to APXinf-robo was on Sep 29, 2026.
What can I use APXinf-robo for?
Deploying a VLA policy on a Jetson robot with low-latency inference. Replacing an OpenPI server with a faster engine without changing the client code. Benchmarking VLA model latency across precisions on Thor, Orin or RTX 4090. Evaluating a π0.5 checkpoint on the LIBERO-10 simulation suite.
How do I install APXinf-robo?
Clone the repo with submodules: git clone --recursive https://github.com/RLinf/APXinf-robo.git Create a virtual environment and install maturin: python3 -m venv .venv && source .venv/bin/activate && pip install maturin Build and install the ApxInf Python binding and Robo package: CARGO_TARGET_DIR=target/wheel maturin build --release --features cuda --auditwheel skip -m apxinf/crates/apxinf-py/Cargo.toml && pip install --force-reinstall target/wheel/wheels/apxinf_py-*.whl && pip install -e "./apxinf/python/apxinf[serving]" --config-settings editable_mode=strict && pip install -e ".[libero,serve]" Run the PI-0.5 benchmark with random weights: python scripts/bench_pi05.py --random-weights --precision bf16 --layer l1 --samples 5
Where can I get APXinf-robo?
The source code and installation instructions are on GitHub: https://github.com/RLinf/APXinf-robo

Summary generated by AI from the project's README. GitHub stats refresh daily.

Related projects