W2SPO — weak model guides strong RL via auxiliary branches
arXiv paper identifies a bottleneck in LLM reasoning RL: on hard tasks, the target model's samples collapse into the same wrong reasoning paths, starving the policy of useful gradient signal.
Proposal: W2SPO, an off-policy method where a weaker auxiliary model explores differently, feeding the strong model with diverse rollouts to escape semantic redundancy.
The framing flips the usual scaling story—cheaper models become a diversity lever, not a dead weight.