An ultra-efficient C99 and CUDA neural inference engine tailored to run the 7-billion parameter Qwen-Image-2.1 diffusion transformer locally on consumer laptop hardware (specifically targeting an AMD Ryzen 7 5800H with an NVIDIA RTX 3060 Mobile 6GB GPU).
Powered by stable-diffusion.cpp and ggml CUDA acceleration, it brings photorealistic local image generation to constrained mobile GPUs.
Key Highlights #
- Strict VRAM Guard ($\le 5.2$ GB): Hard memory ceiling enforcement prevents CUDA out-of-memory driver crashes on 6GB mobile GPUs.
- Sequential GPU Execution: Runs the pipeline stages sequentially on the GPU, managing VRAM footprint dynamically to ensure each stage executes with full GPU acceleration without exceeding mobile memory limits.
- Full Neural VAE Decoder: 238-tensor convolutional decoder with ResNet blocks, GroupNorm, and progressive spatial upsampling directly transforming latent states into high-resolution images.
- Zero Cloud Dependence: 100% offline, local execution with complete data privacy and determinism.