About Me

I am currently a Ph.D. student in Data Science and Artificial Intelligence at The Hong Kong Polytechnic University, supervised by Prof. KC Tan. Previously, I received my M.S. in Applied Statistics from Fudan University and my B.S. in Statistics from Sun Yat-sen University.

My research spans policy diversity in traditional reinforcement learning and reinforcement learning for LLM reasoning (RL for LLM). In policy diversity, I study how to learn and recover diverse behaviors while maintaining policy quality. I also investigate how diverse opponent populations can improve multi-agent training. (ICLR 2023, NeurIPS 2023, ICLR 2025) In RL for LLM, I study how diversity-aware optimization and self-play can strengthen reasoning, and how to reduce redundant reasoning. I also develop robust evaluation methods to assess whether performance gains reflect genuine reasoning ability. (NeurIPS 2025 Spotlight, VAR-MATH, ICML 2026, NeurIPS 2026)

I am always happy to discuss research ideas around RL, LLM Reasoning, Interpretability, and EC + RL. Please feel free to contact me at nigelyaoj@gmail.com.

Experience

ByteDance Seed

Aug 2025 - Jan 2026

Research Intern · LLM Evaluation & Efficient Reasoning

Supervised by Xiongcai Luo

Built LLM evaluation benchmarks and automated scoring pipelines, with statistical analysis to assess evaluation reliability. Developed SLAT to reduce chain-of-thought length by about 50% while maintaining comparable accuracy (ICML 2026).

Tencent AI Lab

Jul 2022 - Apr 2024

Independent Researcher · Reinforcement Learning

Supervised by Dr. Haobo Fu

Studied policy diversity and population-based reinforcement learning for game AI (ICLR 2023, NeurIPS 2023). Developed methods to recover diverse policies from human gameplay and applied them to game AI (ICLR 2025).

Publications

(* equal contribution; † corresponding author)

🤖 RL for LLM

  • [ICML 2026] SLAT: Segment-Level Adaptive Trimming for Efficient CoT Reasoning Jian Yao, Xiongcai Luo, Ran Cheng, KC Tan.

    This work studies how to reduce structural redundancy in chain-of-thought reasoning. It proposes segment-level adaptive trimming to selectively suppress low-utility redundant reasoning segments, improving the accuracy-efficiency trade-off for large reasoning models.

  • [NeurIPS 2026] CasePlay: Self-Play Reinforcement Learning from Case Reports for Medical Reasoning H. Wang, Z. Wang, K. Wu, Jian Yao†, et al..

    This work develops a self-play reinforcement learning framework that alternates between question generation and reasoning on clinical case reports to improve medical reasoning.

  • [NeurIPS 2025 Spotlight] Diversity-Aware Policy Optimization for Large Language Model Reasoning Jian Yao, Ran Cheng, Xingyu Wu, Jibin Wu, KC Tan.

    This work studies the relationship between solution diversity and reasoning potential in LLM reasoning, and proposes a diversity-aware policy optimization method for reinforcement learning training.

  • [arXiv 2025] VAR-MATH: Probing True Mathematical Reasoning in LLMs via Symbolic Multi-Instance Benchmarks Jian Yao, Bowen Zheng, Ran Cheng, KC Tan.

    This work introduces VAR-MATH, a symbolic multi-instance benchmark that probes whether LLM mathematical gains reflect genuine reasoning rather than benchmark-specific overfitting or memorization.

🎮 Diversity in RL

🧩 Others (CV & GNN)

  • [NeurIPS 2022 Spotlight] Self-supervised Amodal Video Object Segmentation Jian Yao, Yuxin Hong, Chiyu Wang, Tianjun Xiao, Tong He, Francesco Locatello, David Wipf, et al..

    This work proposes a self-supervised video segmentation framework for inferring complete object shapes in occluded scenes using temporal information.

  • [NeurIPS 2021] GRIN: Generative Relation and Intention Network for Multi-agent Trajectory Prediction Longyuan Li, Jian Yao, Li Wenliang, Tong He, Tianjun Xiao, Junchi Yan, David P. Wipf, et al..

    This work combines conditional generative modeling with graph neural networks to model agent intentions and social relations for multi-agent trajectory prediction.

Academic Services

  • Reviewer for NeurIPS, ICLR, and ICML, 2023-present
  • Reviewer for AAAI, 2025-present
  • Reviewer for IEEE Transactions on Evolutionary Computation (TEVC)
  • Reviewer for IEEE Transactions on Cognitive and Developmental Systems (TCDS)

Teaching Assistant

  • ENG 2003: Information Technology, The Hong Kong Polytechnic University, Fall 2024
  • COMP 6707: Computational Intelligence, The Hong Kong Polytechnic University, Spring 2025
  • DSAI 4205: Big Data Analysis, The Hong Kong Polytechnic University, Fall 2025 and Spring 2026
Flag Counter