Replymessage unavailable
难绷,我昨天测了一下,bf16的版本效果这样
Intelligence Benchmark Comparison
Mode Sampled Qwen3.6-35B-A3B-bf16 Qwen3.6-35B-A3B-nvfp4
-----------------------------------------------------------------------------------
MMLU Sample 500/14042 88.6% 88.4%
HELLASWAG Sample 200/10042 93.0% 93.5%
TRUTHFULQA Full 817 91.2% 89.1%
HUMANEVAL Full 164 95.7% 93.9%
MBPP Sample 200/500 92.0% 88.5%
LIVECODEBENCH Sample 100/1055 46.0% 34.0%
--- Detail ---
Model: Qwen3.6-35B-A3B-bf16
Benchmark Accuracy Correct Total Time(s) Think
--------------------------------------------------------------
MMLU 88.6% 443 500 8361.4 Yes
HELLASWAG 93.0% 186 200 2609.3 Yes
TRUTHFULQA 91.2% 745 817 10702 Yes
HUMANEVAL 95.7% 157 164 5499 Yes
MBPP 92.0% 184 200 6927.8 Yes
LIVECODEBENCH 46.0% 46 100 19808.4 Yes
Model: Qwen3.6-35B-A3B-nvfp4
Benchmark Accuracy Correct Total Time(s) Think
--------------------------------------------------------------
MMLU 88.4% 442 500 2460.5 Yes
HELLASWAG 93.5% 187 200 678.8 Yes
TRUTHFULQA 89.1% 728 817 3223.2 Yes
HUMANEVAL 93.9% 154 164 1655.8 Yes
MBPP 88.5% 177 200 2913.9 Yes
LIVECODEBENCH 34.0% 34 100 5041.3 Yes