Stage 1
Learn soundness
Reward novel generated tests whose outputs agree with the verified reference solution. This establishes a reliable test generator before adversarial optimization.
Findings of EMNLP 2026
Test Cases Scaling
Two-Stage Reinforcement Learning for Sound and Adversarial Test Generation in Code LLMs
Nanyang Technological University · Skywork AI
Paper overview
Abstract
Reinforcement learning (RL) has substantially advanced code generation with large language models (LLMs) through executable feedback. The feedback for coding problems mainly comes from specific test cases, where high-quality test cases are often scarce since they should be both sound and discriminative. We thus turn to study the auto-generation of test cases using the learned model.
We find this is naturally an adversarial RL problem: the model is expected to generate effective test cases as counterexamples, depending on the solver's current failure modes. We propose Test Cases Scaling (TCS), a two-stage RL framework for effective test generation. Both stages train a test generator from a rolling policy-aligned buffer: Stage 1 generates tests consistent with the reference solution, and Stage 2 restricts the buffer to current failure modes and learns counterexample tests.
Across TACO and LiveCodeBench, TCS improves both pass@1 and inference-time answer selection according to generated tests. We find the learned test generator also enables effective selection among other LLM outputs.
Keywords: Test Cases Scaling (TCS), code LLMs, reinforcement learning, test generation, executable verification, inference-time scaling, answer selection, TACO, and LiveCodeBench.
01
Sound
Generated tests are checked against a ground-truth solution to control false rejection.
02
Adversarial
Tests are rewarded for exposing plausible incorrect programs produced by the current solver.
03
Useful
The learned verifier improves inference-time selection for both TCS and external LLM outputs.
01 Method
A staged verifier curriculum
Reliability first.
Counterexamples second.
Learning adversarial tests from scratch produces sparse and unstable rewards. TCS separates the objective into two stages while keeping training aligned with the evolving solver.
Stage 2
Learn counterexamples
Condition on current incorrect programs and reward tests that pass the reference solution but fail the candidate, targeting the solver's changing error distribution.
Inference
Select with executable evidence
Pool generated tests across sampled candidates, execute every candidate, and choose the program with the highest pass count.
02 Results
TCS-7B · no public tests
Generated tests turn additional samples into better answers.
Pass@1 measures a single generated solution. TCS selection samples multiple candidates and selects among them using self-generated tests.
37.03→48.79
pass@1 → selection with self-generated tests at N=32
+11.76 points
24.09→35.35
pass@1 → selection with self-generated tests at N=16
+11.26 points
All values above are reproduced from the paper's main results. See the paper for complete baselines, public-test settings, model sizes, and evaluation details.
03 Resources
04 Citation
@inproceedings{xu2026tcs,
title = {Two-Stage Reinforcement Learning for Sound and
Adversarial Test Generation in Code {LLM}s},
author = {Jiacheng Xu and Wentao Zhang and Zhiyi Lyu and
Fuxiang Zhang and Chaojie Wang and Yang Liu and Bo An},
booktitle = {Findings of the Association for Computational
Linguistics: {EMNLP} 2026},
year = {2026}
}