# 🖼️ Nunchaku / SVDQuants [![](https://wiki.chenjia.org/uploads/images/gallery/2026-05/scaled-1680-/V4ARov3MJgvCh0m7-image-1779596285309.png)](https://wiki.chenjia.org/uploads/images/gallery/2026-05/V4ARov3MJgvCh0m7-image-1779596285309.png) ### Deep Analysis: Why Optimized INT4 Formats Outperform Q4 (GGUF) Formats in ComfyUI When running large diffusion models like FLUX.1 or SD3/3.5 in ComfyUI, VRAM optimization is critical. While both **Q4** and various **INT4** formats fall under the "4-bit quantization" umbrella, they are fundamentally different. The architectural superiority of optimized INT4 formats does not stem from compressing the file to a smaller size, but rather from **superior data layout arrangement** and the ability to execute calculations **directly on GPU hardware without converting back to FP16**. This document breaks down the low-level technical reasons why INT4 delivers faster inference speeds and significantly better prompt adherence/detail preservation compared to Q4. --- ### 1. Superior Data Layout & Coalesced Memory Alignment The way data is structured and arranged in VRAM dictates how efficiently the GPU can retrieve it. Even at the same bit-width, layout mismatch can severely bottleneck performance. #### The Fragmented Approach of Q4 (Block-wise) * **Structure:** Format families like Q4 (commonly found in GGUF or standard block-wise quantization) arrange data in **fixed blocks** (typically 32 or 64 weights per block). Each block contains a shared FP16 scaling factor (scale) followed by the 4-bit quantized weights. * **The Bottleneck:** When the GPU reads these matrices, it cannot perform a continuous, clean sweep of memory. It must execute a fragmented "hop-and-skip" read pattern to parse the header scale of a block before reading its corresponding 4-bit weights. This interleaved layout breaks memory coalescing, leading to massive inefficiencies in GPU memory bandwidth utilization. #### The Continuous Arrangement of INT4 (Matrix-aligned) * **Structure:** Modern optimized INT4 formats (such as those powered by **Marlin kernels**, **SVDQuant**, or tailored **AWQ** implementations in ComfyUI) decouple the quantization scales from the weights or align them perfectly with the GPU's hardware memory boundaries. * **The Benefit:** Weights are packed into a **Coalesced Memory Layout**. The GPU can stream a massive, continuous block of 4-bit integers in a single memory clock cycle. By achieving 100% continuous data alignment, INT4 maximizes the raw memory bandwidth of your VRAM. --- ### 2. Direct Hardware Execution: Eliminating the FP16 Conversion Overhead The defining differentiator between standard Q4 and optimized INT4 is how they talk to the GPU's execution units. ``` [Q4 Workflow] VRAM ──> Read Block ──> Extract Scale ──> De-quantize to FP16 ──> Tensor Core (FP16 Math) [High Latency] [INT4 Workflow] VRAM ──> Continuous Read ───────────────────────────> Registers ──> Tensor Core (INT4 Math) [Zero Overhead] ``` #### The Q4 Execution Penalty Standard GPU execution cores operate natively on floating-point arithmetic (FP16/BF16). Because Q4 utilizes an irregular, non-standard software packaging layout, the GPU **cannot compute it natively**. * Every time a sampling step occurs, the GPU is forced to execute an explicit **De-quantization (un-packing) step**. * The 4-bit weights must be multiplied by their block scales and cast back into full FP16 values in the shared memory or registers before the Tensor Cores can touch them. This constant on-the-fly conversion creates an immense operational overhead and wastes precious GPU compute cycles. #### The Native INT4 Fast-Path Optimized INT4 implementations bypass the floating-point translation layer entirely. Modern GPU architectures (NVIDIA Ampere, Ada Lovelace, and Blackwell) feature dedicated hardware instructions and specialized Tensor Cores designed for native integer mathematics (such as `DP4A` or native INT4 matrix-multiplication extensions). * **Direct Register Loading:** INT4 data is fetched from VRAM and dumped directly into the GPU registers in its native 4-bit integer form. * **No FP16 Translation:** The GPU executes matrix multiplication directly on the integer data at the hardware level. Removing the intermediate de-quantization step eliminates the math pipeline bottleneck, causing the iteration speed (it/s) to scale dramatically. --- ### 3. Why Native INT4 Preserves Finer Details & Better Prompt Adherence "Detail fidelity" in diffusion models relates directly to the preservation of high-frequency information (fine textures, facial symmetry, text rendering, and complex prompt relationships) handled by the model's Attention mechanisms and Text Encoders. 1. **Accumulation of Rounding Errors in Q4:** When Q4 scales a rigid block of 32 or 64 weights uniformly, any statistical outliers (highly critical structural weights) are forcefully flattened or threshed into a coarse approximation. When this is converted back to FP16 on-the-fly, the resulting precision errors cascade across dozens of sampling steps. In ComfyUI, this manifests as muddy skin textures, malformed hands, distorted eyes, or a failure to follow intricate, multi-subject prompts. 2. **Precision Bit-Conservation in INT4:** Because optimized INT4 formats employ hardware-aligned integer math combined with activation-aware layout mapping (e.g., safeguarding activation outliers beforehand), the structural weights remain physically uncorrupted by lossy mid-flight floating-point conversions. The high-frequency detail channels remain mathematically pure from the first sampling step to the final VAE decode. --- ### Technical Summary Matrix | Architectural Feature | Q4 Format (Block-wise / GGUF) | Optimized INT4 Format (Marlin / SVDQuant) | | :--- | :--- | :--- | | **VRAM Layout Packing** | Interleaved headers and weights; prone to memory fragmentation. | Coalesced, continuous matrix blocks; aligned to GPU warp/thread layouts. | | **GPU Processing Pipeline** | Mandatory runtime **De-quantization to FP16** before execution. | **Direct Register Injection**; calculated natively using Integer Tensor Cores. | | **Baking / Scale Strategy** | Blind block constraints (forced grouping regardless of weight importance). | Advanced layout alignment; preserves outlier weight integrity natively. | | **ComfyUI Impact** | Slower generation rates due to translation latency; degradation in micro-details. | Maximum VRAM bandwidth saturation; flawless prompt fidelity and sharp textures. | --- ### Conclusion The shift toward **INT4** inside ComfyUI represents an evolution from *generic file compression* (Q4) to *hardware-level compute optimization* (INT4). By realigning the data structure to match the physical architecture of modern GPUs and eliminating the computational tax of FP16 conversion, optimized INT4 establishes itself as the superior standard for running ultra-large scale generative models at maximum speed without compromising a single pixel of detail.