How to Deploy Qwen3-VL-4B with vLLM

# Create a working directory (choose your preferred location)

# Create virtual environment with Python 3.12
# Make sure you have Python 3.12 installed: sudo apt install python3.12 python3.12-venv
python3.12 -m venv venv

# Activate virtual environment
source venv/bin/activate

# Verify Python version (should be 3.12.x)
python --version

# Upgrade pip
pip install --upgrade pip

# Install vLLM with video support (this takes ~5-10 minutes)
pip install vllm
pip install 'vllm[video]'

# Make sure you're in your working directory and activate virtual environment
source venv/bin/activate

# Launch vLLM serve (maximum context for solo deployment)
vllm serve Qwen/Qwen3-VL-4B-Instruct \
  --dtype bfloat16 \
  --max-model-len 65536 \
  --max-num-batched-tokens 65536 \
  --gpu-memory-utilization 0.9 \
  --port 8000

curl http://localhost:8000/v1/models

{
  "object": "list",
  "data": [{
    "id": "Qwen/Qwen3-VL-4B-Instruct",
    "object": "model",
    "created": 1769822588,
    "owned_by": "vllm",
    "root": "Qwen/Qwen3-VL-4B-Instruct",
    "parent": null,
    "max_model_len": 65536
  }]
}

curl http://localhost:8000/v1/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "Qwen/Qwen3-VL-4B-Instruct",
    "prompt": "Hello, how are you?",
    "max_tokens": 50
  }'

# Encode video to base64
VIDEO_BASE64=$(base64 -w 0 /path/to/your/video.mp4)

# Create request file
cat > /tmp/video_request.json << EOF
{
  "model": "Qwen/Qwen3-VL-4B-Instruct",
  "messages": [{
    "role": "user",
    "content": [
      {
        "type": "video_url",
        "video_url": {
          "url": "data:video/mp4;base64,$VIDEO_BASE64"
        }
      },
      {
        "type": "text",
        "text": "Describe the user actions in this video in a sequential list."
      }
    ]
  }],
  "max_tokens": 1024
}
EOF

# Time the request
time curl http://localhost:8000/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d @/tmp/video_request.json

A man in a red jersey is standing on a grassy field. He is clapping his hands
together and appears to be speaking or shouting. He turns his head to the left,
still clapping, and then turns back to face forward, continuing to clap.