ThinkingCap-Qwen3.8

NVFP4 weight-only (NVFP4A16, compressed-tensors nvfp4-pack-quantized, 16-element FP4 groups with FP8 scales), built by llm-compressor. The two GDN projections vLLM fuses into one GEMM (in_proj_qkv, in_proj_z) share one global scale: vLLM keeps a single scale for the fused GEMM, so separate scales would dequantize one of the two too small. Runs on Hopper (Marlin) and Blackwell.

Built from bottlecapai/ThinkingCap-Qwen3.8-27B (bf16). Vision tower, MTP head, lm_head and the GDN in_proj_a / in_proj_b projections stay bf16 (recipe.yaml). Serve with vLLM 0.29 (--trust-remote-code not needed).

Expected performance

Paired comparison with the bf16 source on the full quantization plan: both builds answer the same questions with the same seeds — RealWorldQA 765 questions × 2 seeds (images), GPQA-Diamond 198 × 4, MMLU-Pro 1,500 × 1 (a fixed slice), IFBench 300 × 2, AA-LCR 100 × 1 (long-document prompts, graded by Gemma-4-26B-A4B-it with thinking off). Thinking at the chat template's default reasoning effort (xhigh), sampled decoding (temperature 1.0, top_p 0.95, top_k 20, min_p 0.0), 65,536-token generation cap. Both builds served by vLLM 0.29.0 on an H200.

Δ accuracy is NVFP4 minus bf16 on the same answers; its interval treats the question as the unit (seeds averaged per question first). Tokens are completion tokens (reasoning plus answer). Bold: interval excludes zero.

Mean tokens fall significantly on IFBench and AA-LCR with no significant accuracy change, but the medians there rise 7–9%: the drop is fewer very long answers, not uniformly shorter ones.

Throughput — one RTX PRO 6000 Blackwell, vLLM 0.29.0 (Marlin FP4 kernel for this build), synthetic prompts of 1,024 tokens with 512 generated (last column: 32,768-token prompts, 128 generated), end-of-sequence ignored, prefix caching off, --max-num-seqs 64. Aggregate output tokens/s, median time to first token in ms in parentheses. With 32k-token prompts this build is no faster than bf16 and its first token comes later.

Decode speed and MTP self-speculative decoding (MMLU-Pro) — 32 questions × 1 seed, vLLM 0.26 on one H200, 16 concurrent requests

Where to find us

Need even more efficiency? The open release is production-ready. Our enterprise versions go further — fewer thinking tokens still, tuned to your workload, at matched accuracy on your own tasks. Built for AI labs, inference providers and enterprises running models at scale. Deployed on your infrastructure, or in the cloud and region you choose. Talk to our team

License

ThinkingCap: PolyForm Small Business 1.0.0 + BottleCap personal-use grant (see LICENSE).

Upstream Qwen materials: Apache-2.0 (see NOTICE).

Commercial license: contact BottleCap AI.

Citation

If you use this model, please cite:

@misc{ThinkingCap-Qwen3.8-27B,
  title     = {bottlecapai/ThinkingCap-Qwen3.8-27B},
  author    = {Osusky, Adam and Lindauer, Jan and Jirkovsky, Adam and Mihal, Filip and Platek, Ondrej and Herel, David and Ihnatchenko, Luka and Bartek, Vojtech and Jirak, Jiri and Kubista, Daniel and Krus, Frantisek and Mikolov, Tomas},
  year      = {2026},
}