VinRobotics · AIWintermuteAI · Jul 7, 2026
diff --git a/docs/DOCKER.md b/docs/DOCKER.md
@@ -0,0 +1,189 @@
+# Docker evaluation workflow for vla.cpp
+
+This document describes the Docker Compose evaluation stack, which runs the
+vla.cpp inference server and the Python simulation client in separate
+containers.
+
+| Container | Image | Purpose |
+|-----------|-------|---------|
+| `server` | upstream `Dockerfile` | C++ `vla-server` daemon (CUDA or CPU) |
+| `client` | `eval/Dockerfile.client` | Python simulation environment (MuJoCo, robosuite, lerobot) |
+
+---
+
+## Quick start
+
+### Prerequisites
+
+- [Docker Compose](https://docs.docker.com/compose/) v2.24+
+- NVIDIA GPU with proprietary driver ≥ 535. GPU access uses **CDI**
+  (`devices: - nvidia.com/gpu=all`) — no `nvidia-container-toolkit` needed.
+  See [CUDA GPU access](#cuda-gpu-access) for details.
+
+### 1. Download model GGUF files
+
+```bash
+docker compose -f eval/docker-compose.yml build client
+docker compose -f eval/docker-compose.yml run --no-deps --rm client \
+    huggingface-cli download vrfai/smolvla-libero-gguf --local-dir /models
+```
+
+> `--no-deps` skips building the server image, which isn't needed for downloads.
+
+Models are mounted into both containers at `/models`.
+
+### 2. Build the images
+
+```bash
+docker compose -f eval/docker-compose.yml build
+```
+
+Build args accepted by the server `Dockerfile`:
+
+| Arg | Default | Notes |
+|-----|---------|-------|
+| `BACKEND` | `cuda` | `cuda` or `cpu` |
+| `CUDA_ARCH` | `120` | Blackwell; `89` for RTX40, `87` for Orin, `86` for RTX30 |
+| `BASE_IMAGE` | `nvidia/cuda:12.9.1-devel-ubuntu24.04` | — |
+| `JOBS` | `nproc` | Lower if nvcc segfaults on flash-attn kernels |
+
+Override via e.g. `docker compose -f eval/docker-compose.yml build --build-arg CUDA_ARCH=89 server`.
+
+### 3. Start the server
+
+```bash
+docker compose -f eval/docker-compose.yml up -d server
+docker compose -f eval/docker-compose.yml logs server
+# … vla-server: bound to tcp://*:5555. ready.
+```
+
+The default `command` in `eval/docker-compose.yml` starts SmolVLA for LIBERO.
+For other models (e.g. GR00T-N1.7, no mmproj needed):
+
+```bash
+docker compose -f eval/docker-compose.yml run --rm server \
+    --bind tcp://*:5555 /models/gr00tn1d7-libero.gguf
+```
+
+### 4. Run an evaluation episode
+
+```bash
+docker compose -f eval/docker-compose.yml run --rm client \
+    python eval/client/run_sim_client_direct.py \
+        --task libero_object --task-id 0 --n-episodes 1 \
+        --output-dir /tmp/libero_outputs --arch smolvla \
+        --vla-addr tcp://server:5555
+```
+
+Or drop into an interactive shell:
+
+```bash
+docker compose -f eval/docker-compose.yml run --rm client
+root@...:/workspace/vla.cpp# python eval/client/run_sim_client_direct.py \
+    --task libero_object --task-id 0 --n-episodes 1 \
+    --output-dir /tmp/libero_outputs --arch smolvla \
+    --vla-addr tcp://server:5555
+```
+
+Results (videos, summary) are written to `/tmp/libero_outputs` on the host.
+
+**Example output (RTX 5060 Ti, CUDA arch 120):**
+
+```
+vla-cpp-direct[arch=smolvla]: connected to tcp://server:5555
+- Step 220: reward=1.00, done=True, truncated=False
+- Episode finished after 220 steps.  Final reward: 1.00
+- Success rate: 100.00%  (1/1)
+- Average inference time per step: 116.45 ms
+```
+
+---
+
+## Configuration reference
+
+### Volumes
+
+| Host / Volume | Container mount | Purpose |
+|---------------|----------------|---------|
+| `/tmp/smolvla-models` | `client:/models` (rw), `server:/models:ro` | GGUF model files |
+| `/tmp/libero_outputs` | `client:/tmp/libero_outputs` | Eval videos & summaries |
+| `hf-cache` (named) | `client:/root/.cache/huggingface` | HuggingFace tokenizer cache |
+
+### Ports
+
+| Service | Host | Container |
+|---------|------|-----------|
+| server  | `5555` | `5555` |
+
+### Network
+
+Both services share the default Compose network. The client reaches the server
+via hostname `server`.
+
+### CUDA GPU access
+
+The server uses CDI (`devices: - nvidia.com/gpu=all`). This works without
+`nvidia-container-toolkit` as long as:
+1. The NVIDIA proprietary driver is installed (≥ 535).
+2. A CDI-enabled container runtime is available (containerd ≥ 1.7,
+   cri-o ≥ 1.29, or Docker with `nvidia-ctk` from `nvidia-container-toolkit`
+   ≥ 1.15 to generate `/etc/cdi/nvidia.yaml`).
+
+---
+
+## Running without Docker Compose
+
+### Server only
+
+```bash
+docker build -t vla-cpp-server \
+    --build-arg BACKEND=cuda --build-arg CUDA_ARCH=120 .
+
+# CDI
+docker run --rm --device nvidia.com/gpu=all -p5555:5555 \
+    -v /tmp/smolvla-models:/models:ro \
+    vla-cpp-server --bind tcp://*:5555 /models/model.gguf
+
+# nvidia-container-toolkit
+docker run --rm --gpus all -p5555:5555 \
+    -v /tmp/smolvla-models:/models:ro \
+    vla-cpp-server --bind tcp://*:5555 /models/model.gguf
+```
+
+### Client only
+
+```bash
+docker build -t vla-cpp-client -f eval/Dockerfile.client .
+docker run --rm -it --network host \
+    -v /tmp/libero_outputs:/tmp/libero_outputs \
+    vla-cpp-client
+# Inside: connect to server at localhost:5555
+```
+
+---
+
+## Known issues
+
+| Issue | Workaround |
+|-------|-----------|
+| `Unsupported gpu architecture 'compute_120'` with CUDA < 12.8 | Use CUDA 12.8+ for `sm_120`, or set `CUDA_ARCH=89` for RTX40-series compatibility |
+| NumPy 2.x: `module 'numpy' has no attribute 'core'` | `Dockerfile.client` pins `numpy==1.26.4` and patches accelerate |
+| `lerobot` pulls GPU torch | `Dockerfile.client` re-pins `torch==2.5.1` (CPU) after installing lerobot |
+| LIBERO data files not found | Editable install (`-e`) keeps `bddl_files/` / `init_files/` / `assets/` accessible at runtime |
+| LIBERO hangs on first import (dataset path prompt) | `echo "N" \| python3 -c "import libero.libero"` pre-seeds `~/.libero/config.yaml` |
+| `pandas` segfaults on import | Pin `pandas==2.0.3` (last NumPy 1.x-compatible release) |
+| MuJoCo 3.x: robosuite init fails | Pin `mujoco<3.0` (2.3.7 known-good) |
+| `nvidia-container-toolkit` not installed | Use CDI (`devices: - nvidia.com/gpu=all`) instead of `runtime: nvidia` |
+
+---
+
+## Summary
+
+The Docker Compose evaluation stack provides a reproducible two-container
+workflow for vla.cpp:
+
+1. **Server** — upstream `Dockerfile`, compiles `vla-server` with GPU support.
+2. **Client** — `eval/Dockerfile.client`, Python simulation stack with pinned
+   dependency versions (NumPy 1.x, MuJoCo 2.x, Pandas 2.0.x).
+3. **CDI** eliminates the `nvidia-container-toolkit` dependency for GPU access.
+4. **First-step overhead** (~35 s CUDA graph warmup) occurs once per process.
diff --git a/eval/Dockerfile.client b/eval/Dockerfile.client
@@ -0,0 +1,126 @@
+# vla.cpp evaluation client container.
+#
+# Contains the Python simulation environment (MuJoCo, robosuite, lerobot) and
+# the eval scripts needed to drive vla-server from a LIBERO / SimplerEnv
+# episode.  No CUDA, no C++ build — this is purely a Python runtime.
+#
+# Usage (with the upstream server Dockerfile):
+#
+#   # Terminal 1 — build & start the server:
+#   docker build -t vla-cpp-server .                              # upstream Dockerfile
+#   docker run --gpus all -p5555:5555 -v $PWD/models:/models \
+#       vla-cpp-server --bind tcp://*:5555 /models/model.gguf
+#
+#   # Terminal 2 — build & run the client:
+#   docker build -t vla-cpp-client -f eval/Dockerfile.client .
+#   docker run --rm -it --network host \
+#       -v /tmp/libero_outputs:/tmp/libero_outputs \
+#       vla-cpp-client
+#   root@...:/workspace/vla.cpp# python eval/client/run_sim_client_direct.py \
+#       --task libero_object --task-id 0 --n-episodes 1 \
+#       --output-dir /tmp/libero_outputs --arch smolvla \
+#       --vla-addr tcp://localhost:5555
+#
+# Or with docker-compose (see eval/docker-compose.yml).
+
+FROM ubuntu:22.04
+
+ENV DEBIAN_FRONTEND=noninteractive \
+    TZ=UTC \
+    LANG=C.UTF-8 \
+    LC_ALL=C.UTF-8 \
+    PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python \
+    VLA_CPP_PROTO=/workspace/vla.cpp/src/serving/vla.proto
+
+# System dependencies: build tools, protobuf, ZMQ, EGL (for MuJoCo offscreen)
+RUN apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    cmake \
+    git \
+    curl \
+    ca-certificates \
+    libzmq3-dev \
+    libprotobuf-dev \
+    protobuf-compiler \
+    python3.10 \
+    python3.10-venv \
+    python3.10-dev \
+    python3-pip \
+    libegl-dev \
+    && rm -rf /var/lib/apt/lists/*
+
+# Make python3.10 the default
+RUN update-alternatives --install /usr/bin/python3 python3 /usr/bin/python3.10 1 && \
+    ln -sf /usr/bin/python3 /usr/local/bin/python
+RUN curl -sS https://bootstrap.pypa.io/get-pip.py | python3.10
+
+WORKDIR /workspace/vla.cpp
+COPY . .
+
+# ---------------------------------------------------------------------------
+# Python dependencies — order matters to avoid version conflicts
+# ---------------------------------------------------------------------------
+
+# Pin foundational deps first so nothing pulls numpy 2.x or mujoco 3.x
+RUN pip3 install --no-cache-dir numpy==1.26.4
+RUN pip3 install --no-cache-dir "mujoco<3.0"
+
+# PyTorch CPU-only (the server handles GPU inference)
+RUN pip3 install --no-cache-dir torch==2.5.1 --index-url https://download.pytorch.org/whl/cpu
+RUN pip3 install --no-cache-dir torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cpu
+
+# LIBERO — clone and install.
+# We use editable install (following eval/sim/libero/setup_libero.sh) so that
+# the source tree with bddl_files/, init_files/, assets/ stays accessible at
+# runtime.  The first import seeds ~/.libero/config.yaml via the script's
+# "echo N" trick, avoiding the interactive dataset-path prompt.
+#
+# The CMAKE_POLICY_VERSION_MINIMUM env-var is needed by egl-probe (a transitive
+# dep of robomimic/robosuite) which requests cmake_minimum_required < 3.5.
+RUN CMAKE_POLICY_VERSION_MINIMUM=3.5 pip3 install --no-cache-dir --upgrade pip setuptools && \
+    git clone https://github.com/Lifelong-Robot-Learning/LIBERO.git /tmp/LIBERO && \
+    pip3 install --no-cache-dir -r /tmp/LIBERO/requirements.txt && \
+    CMAKE_POLICY_VERSION_MINIMUM=3.5 pip3 install --no-cache-dir \
+        -e /tmp/LIBERO --config-settings editable_mode=compat && \
+    rm -rf /root/.libero && \
+    echo "N" | python3 -c "import libero.libero" 2>/dev/null || true
+
+# lerobot — install with dependencies but immediately re-pin torch so it is
+# not upgraded from our pinned CPU-only version.
+RUN pip3 install --no-cache-dir lerobot==0.4.3 && \
+    pip3 install --no-cache-dir torch==2.5.1 --index-url https://download.pytorch.org/whl/cpu && \
+    pip3 install --no-cache-dir torchvision==0.20.1 --index-url https://download.pytorch.org/whl/cpu
+
+# Remaining client dependencies
+RUN pip3 install --no-cache-dir \
+    transformers==4.51.3 \
+    gymnasium==0.29.1 \
+    zmq \
+    msgpack==1.1.0 \
+    msgpack-numpy==0.4.8 \
+    pillow \
+    imageio \
+    tqdm \
+    pydantic \
+    av \
+    tianshou==0.5.1 \
+    tyro \
+    pandas==2.0.3 \
+    dm_tree \
+    einops==0.8.1 \
+    albumentations==1.4.18 \
+    pyarrow==12.0.1 \
+    diffusers==0.30.1 \
+    huggingface-hub
+
+# Re-pin numpy to 1.x — later deps (albumentations, tianshou, etc.) pull in 2.x
+RUN pip3 install --no-cache-dir numpy==1.26.4
+
+# Apply the accelerate monkey-patch for numpy compat
+RUN sed -i 's/np_core = np._core if is_numpy_available("2.0.0") else np.core/np_core = np.core/' \
+    /usr/local/lib/python3.10/dist-packages/accelerate/utils/other.py 2>/dev/null || true
+
+# Clean up
+RUN rm -rf /root/.cache/pip
+
+CMD ["bash"]
diff --git a/eval/docker-compose.yml b/eval/docker-compose.yml
@@ -0,0 +1,78 @@
+# vla.cpp two-container evaluation stack.
+#
+#   server  — vla-server (built with the upstream Dockerfile)
+#   client  — Python simulation environment (eval/Dockerfile.client)
+#
+# Usage:
+#   docker compose -f eval/docker-compose.yml build       # builds both images
+#   docker compose -f eval/docker-compose.yml up -d server # start server
+#   docker compose -f eval/docker-compose.yml run --rm client  # interactive shell
+#
+# Inside the client shell:
+#   python eval/client/run_sim_client_direct.py \
+#       --task libero_object --task-id 0 --n-episodes 1 \
+#       --output-dir /tmp/libero_outputs --arch smolvla \
+#       --vla-addr tcp://server:5555
+#
+# The server is reachable at hostname "server" (Docker Compose DNS).
+
+services:
+
+  # --------------------------------------------------------------------------
+  # vla-server — C++ inference daemon (upstream Dockerfile)
+  # Build args:
+  #   BACKEND=cuda|cpu   (default: cuda)
+  #   CUDA_ARCH          (default: 120, use 89 for RTX40, 87 for Orin, etc.)
+  # --------------------------------------------------------------------------
+  server:
+    build:
+      context: ..
+      dockerfile: Dockerfile
+      args:
+        BACKEND: cuda
+        CUDA_ARCH: "120"
+    image: vla-cpp-server
+    container_name: vla-cpp-server
+    devices:
+      - nvidia.com/gpu=all   # CDI
+    volumes:
+      # Mount GGUF model files (read-only)
+      - /tmp/smolvla-models:/models:ro
+    environment:
+      - CUDA_VISIBLE_DEVICES=0
+    ports:
+      - "5555:5555"
+    # For SmolVLA / π0 pass mmproj + model GGUF.
+    # For baked-in-vision models (BitVLA, GR00T, Evo-1) omit mmproj.
+    command:
+      - --bind
+      - tcp://*:5555
+      - /models/mmproj-smolvla-libero.gguf
+      - /models/smolvla-libero.gguf
+
+  # --------------------------------------------------------------------------
+  # Client — Python simulation environment (eval/Dockerfile.client)
+  # --------------------------------------------------------------------------
+  client:
+    build:
+      context: ..
+      dockerfile: eval/Dockerfile.client
+    image: vla-cpp-client
+    container_name: vla-cpp-client
+    stdin_open: true
+    tty: true
+    depends_on:
+      - server
+    volumes:
+      # Output directory for eval videos / summaries
+      - /tmp/libero_outputs:/tmp/libero_outputs
+      # Model download destination (also used by server)
+      - /tmp/smolvla-models:/models
+      # Cache for HuggingFace tokenizers etc.
+      - hf-cache:/root/.cache/huggingface
+    environment:
+      - PROTOCOL_BUFFERS_PYTHON_IMPLEMENTATION=python
+      - VLA_CPP_PROTO=/workspace/vla.cpp/src/serving/vla.proto
+
+volumes:
+  hf-cache: