Show HN：我将 DeepSeek V4 Flash 缩小到 57GB，它在我的 Mac 上编写了一个编译器
我专门构建了一个DeepSeek V4 Flash 0731的包（最初284B总参数，13B活动），保留推理、工具调用和编码功能：https://huggingface.co/steadfastgaze/DeepSeek-V4-Flash-0731-...我让它编写一个针对ARM64的最小C编译器，然后用Fibonacci和FizzBuzz程序测试结果，不到1小时就成功了，完整记录位于：https://youtu.be/XiwSilmV8B0 您可以使用我的引擎在 Silicon Mac 上运行它 https://github.com/steadfastgaze/MoEspresso ，而为获得此结果而开发的核心库之一可在…
查看英文原文
Show HN: I shrank DeepSeek V4 Flash to 57GB and it wrote a compiler on my Mac
I built a specialized package of DeepSeek V4 Flash 0731 (originally 284B total parameters, 13B active), preserving reasoning, tool calling and coding capabilities: https://huggingface.co/steadfastgaze/DeepSeek-V4-Flash-0731-... I let it write a minimal C compiler targeting ARM64, then test the result with Fibonacci and FizzBuzz programs, and it succeeded in less than 1 hour, with the full recording at: https://youtu.be/XiwSilmV8B0 You can run it on Silicon Macs with my engine https://github.com/steadfastgaze/MoEspresso , while one of the core libraries developed to obtain this result is available at https://github.com/steadfastgaze/mlx-iqk . The above recording was on a 128GB memory MacBook M3 Max, but you can also run it on 32GB MacBooks with a very usable context (128K tokens) and projected 5 tok/s. I did try it on a fanless 16GB memory MacBook Air M1 (1.39 tok/s), but unfortunately the available context was very small. How: - First, efficient quantisation: mlx-iqk takes advantage of IQ_K tensor encoding, more efficient than the ones available via llama.cpp or barebones MLX, originally designed by Iwan Kawrakow - I also changed the layout to a k-contiguous one, to make it faster, at least in this Metal setup. - Second, expert pruning: each of the 40 learned-router layers had 256 experts, and not all of them are equally important for the coding use cases. I removed 80B parameters - this is a known technique called REAP, shared at https://www.cerebras.ai/blog/reap . - Third: balancing the cheapest IQ1_S_R4 tensor encoding (~1.5 bits per weight), selectively promoting projections to IQ2_KS or IQ2_K where the measured error reduction justified the bytes. One of the main ideas was not only to save relevant knowledge, but also to not make it forget how to... stop thinking, how to use reasoning. In the first experiments, it would sometimes reason for thousands of tokens without closing its thinking section, or it would go in loops. Then I solved this by heavily weighting tool-calling traces and structured reasoning in the calibration mix.
Show HN: I shrank DeepSeek V4 Flash to 57GB and it wrote a compiler on my Mac
I built a specialized package of DeepSeek V4 Flash 0731 (originally 284B total parameters, 13B active), preserving reasoning, tool calling and coding capabilities:<p><a href="https://huggingface.co/steadfastgaze/DeepSeek-V4-Flash-0731-Coder-56.8GB-MoEspressoV2" rel="nofollow">https://huggingface.co/steadfastgaze/DeepSeek-V4-Flash-0731-...</a><p>I let it write a minimal C compiler targeting ARM64, then test the result with Fibonacci and FizzBuzz programs, and it succeeded in less than 1 hour, with the full recording at:<p><a href="https://youtu.be/XiwSilmV8B0" rel="nofollow">https://youtu.be/XiwSilmV8B0</a><p>You can run it on Silicon Macs with my engine <a href="https://github.com/steadfastgaze/MoEspresso" rel="nofollow">https://github.com/steadfastgaze/MoEspresso</a>, while one of the core libraries developed to obtain this result is available at <a href="https://github.com/steadfastgaze/mlx-iqk" rel="nofollow">https://github.com/steadfastgaze/mlx-iqk</a>.<p>The above recording was on a 128GB memory MacBook M3 Max, but you can also run it on 32GB MacBooks with a very usable context (128K tokens) and projected 5 tok/s. I did try it on a fanless 16GB memory MacBook Air M1 (1.39 tok/s), but unfortunately the available context was very small.<p>How: - First, efficient quantisation: mlx-iqk takes advantage of IQ_K tensor encoding, more efficient than the ones available via llama.cpp or barebones MLX, originally designed by Iwan Kawrakow - I also changed the layout to a k-contiguous one, to make it faster, at least in this Metal setup.<p>- Second, expert pruning: each of the 40 learned-router layers had 256 experts, and not all of them are equally important for the coding use cases. I removed 80B parameters - this is a known technique called REAP, shared at <a href="https://www.cerebras.ai/blog/reap" rel="nofollow">https://www.cerebras.ai/blog/reap</a>.<p>- Third: balancing the cheapest IQ1_S_R4 tensor encoding (~1.5 bits per weight), selectively promoting projections to IQ2_KS or IQ2_K where the measured error reduction justified the bytes.<p>One of the main ideas was not only to save relevant knowledge, but also to not make it forget how to... stop thinking, how to use reasoning. In the first experiments, it would sometimes reason for thousands of tokens without closing its thinking section, or it would go in loops.<p>Then I solved this by heavily weighting tool-calling traces and structured reasoning in the calibration mix.
- 来源
- Show HN · show-hn
- 证据角色
- project_and_comment_discovery
- 时间状态
- known
- 用户判决
- 未判