Jiacheng XuHomepage

Findings of EMNLP 2026

TCSTwo-Stage Reinforcement Learning for Sound and Adversarial Test Generation in Code LLMs

Jiacheng Xu, Wentao Zhang, Zhiyi Lyu, Fuxiang Zhang, Chaojie Wang, Yang Liu, and Bo An

Nanyang Technological University · Skywork AI

Better tests. Better choices.

Learn tests that are both correct and good at exposing bugs,
then use them to choose among generated programs.

TCS alternates solver and test-generator training with a policy-aligned buffer, moving from sound tests to candidate-conditioned counterexamples.
Reliability first, counterexamples second. TCS keeps test generation aligned with the evolving solver. View full-size figure ↗

01 / The method

A test must be right before it can be useful.

TCS separates learning reliable tests from learning to expose the solver’s mistakes.

Stage 1

Learn soundness.

Reward novel tests whose expected outputs agree with a verified reference solution. Establish reliability before adversarial training.

Stage 2

Target current failures.

Condition on incorrect programs from the current solver. Reward tests that pass the reference solution and fail the candidate.

Inference

Select with execution.

Pool generated tests, execute sampled programs, and select the candidate with the highest test pass count.

02 / The evidence

Use additional samples to select better answers.

TCS-7B · no public tests. These comparisons move from single-sample generation to selection among multiple candidates; they use different inference budgets.

LiveCodeBench

+11.76 pppass@1 → selection at N = 32
37.03 → 48.79

TACO

+11.26 pppass@1 → selection at N = 16
24.09 → 35.35

Beyond the trained solver

Cross-modelTCS-generated tests also select
among other LLMs’ outputs

The full paper reports baselines, public-test settings, model sizes, and evaluation details. Training uses reference solutions; generated tests at inference are not a proof of program correctness.

03 / Interpretation

Execution provides evidence, not a guarantee.

TCS relies on reference solutions during training to assess soundness. At inference, generated tests can still miss bugs or contain incorrect expected outputs. Selection also costs additional program samples and executions.

In the authors’ words

Abstract

Reinforcement learning (RL) has substantially advanced code generation with large language models (LLMs) through executable feedback. The feedback for coding problems mainly comes from specific test cases, where high-quality test cases are often scarce since they should be both sound and discriminative. We thus turn to study the auto-generation of test cases using the learned model.

We find this is naturally an adversarial RL problem: the model is expected to generate effective test cases as counterexamples, depending on the solver's current failure modes. We propose Test Cases Scaling (TCS), a two-stage RL framework for effective test generation. Both stages train a test generator from a rolling policy-aligned buffer: Stage 1 generates tests consistent with the reference solution, and Stage 2 restricts the buffer to current failure modes and learns counterexample tests.

Across TACO and LiveCodeBench, TCS improves both pass@1 and inference-time answer selection according to generated tests. We find the learned test generator also enables effective selection among other LLM outputs.

Explore the work

Resources

Build on this work

Citation

@inproceedings{xu2026tcs,
  title     = {Two-Stage Reinforcement Learning for Sound and
               Adversarial Test Generation in Code {LLM}s},
  author    = {Jiacheng Xu and Wentao Zhang and Zhiyi Lyu and
               Fuxiang Zhang and Chaojie Wang and Yang Liu and Bo An},
  booktitle = {Findings of the Association for Computational
               Linguistics: {EMNLP} 2026},
  year      = {2026}
}

Figures reproduced from the arXiv version by Jiacheng Xu and coauthors, under CC BY 4.0.