Show HN: NanoRL – RL training for LLMs in ~1,800 lines
The smallest async RL trainer I could write: one loop that runs REINFORCE on CartPole on a laptop and async GRPO on a cluster (e.g. 8xH100 trainer, 8 vLLM workers, ran as a [SkyPilot job group]( https://docs.skypilot.ai/en/latest/examples/job-groups.html ) on k8s ). All without Ray or TRL or DeepSpeed etc., workers talk to the trainer over stdlib HTTP.
Show HN: NanoRL – RL training for LLMs in ~1,800 lines
The smallest async RL trainer I could write: one loop that runs REINFORCE on CartPole on a laptop and async GRPO on a cluster
(e.g. 8xH100 trainer, 8 vLLM workers, ran as a [SkyPilot job group]( https://docs.skypilot.ai/en/latest/examples/job-groups.html ) on k8s ). All without Ray or TRL or DeepSpeed etc., workers talk to the trainer over stdlib HTTP.