🤖 Qwen3.6 Local agent test cases

Common user GPU only have 8-16G VRAM.
When the model not fully loaded to VRAM, they have to offload into RAM and processing data with CPU.
This cause local AI system super slow, token generation speed only reach 1-5 token/s.
This paper is writing about test cases and looking for the best configuration for self-host agent.

I run llama.cpp with Qwen3.6-35B-A3B-MTP-GGUF (IQ4_XS).
Then public the API key to use vscode + pi.dev agent.

Test

This caseis 1:

my compose to run llama.cpp:

services:
  llama-cpp:
    image: sandichhuu/llama-cpp:cu132
    container_name: llama-cpp
    ports:
      - 8080:8080
    environment:
      - GGML_CUDA_GRAPH_OPT=1
      - LD_LIBRARY_PATH=/usr/local/nvidia/lib64:/usr/local/nvidia/lib:/usr/lib/x86_64-linux-gnu
    privileged: true
    ipc: host
    volumes:
      - /mnt/data/files/models:/app/models
    command: >
      -m /app/models/unsloth/Qwen3.6-35B-A3B-MTP-GGUF/Qwen3.6-35B-A3B-UD-IQ4_NL.gguf
      --mmproj /app/models/unsloth/Qwen3.6-35B-A3B-MTP-GGUF/mmproj-BF16.gguf
      --port 8080 --host 0.0.0.0
      -t 12 -c 262144 --parallel 1 -fa on
      --cache-type-k q4_0 --cache-type-v q4_0 
      --n-cpu-moe 35 --no-context-shift
      -b 4096 -ub 2048 
      --no-mmap --direct-io --fit off --jinja --no-cache-prompt --cache-ram 0
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all
              capabilities:
                - gpu
    restart: unless-stopped

Test case 1:

The llama.cpp process consume 16G of RAM and 11G of VRAM.
And graph help understand how it's working.

On the case trigger data layers on RAM -> CPU will working. then trigger data layers on VRAM -> GPU will working.
This mean