Bowen Qin
National University of Singapore (NUS), Singapore. eyuansu71@gmail.com
I am a 1st-year Ph.D. student at the National University of Singapore (NUS), advised by Prof. Yao Lu.
Previously, I was a researcher at the Beijing Academy of Artificial Intelligence (BAAI), specializing in the evaluation, alignment, and code intelligence of large language models (LLMs). Previously, I obtained my master’s degree with top honors from the Shenzhen Institute of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), under the guidance of Prof. Min Yang in 2023.
news
| Sep 17, 2026 | We introduced ImpossibleRubrics, a benchmark for testing whether generated rubrics reward unsupported answers over honest responses to impossible tasks. Announcement. |
|---|---|
| Aug 21, 2026 | I shared an analysis of context compaction across roughly 400,000 coding-agent turns, examining tool errors and tool use around compaction. Discussion. |
| May 23, 2026 | I introduced LogDx-CI, a benchmark comparing log-reduction tools for LLM diagnosis of real CI failures. Announcement. |
| May 13, 2026 | We introduced Agent Retrieval Bench to evaluate the retrieval of task-relevant repository files for coding agents. Announcement. |
recent publications
- arXivBIRD-RL: Scaling Agentic Reinforcement Learning over Stateful Data-Centric EnvironmentsarXiv preprint arXiv:2610.29426, Oct 2026
- arXivORCA: Evaluating LLMs on Data Science Code TranslationarXiv preprint arXiv:2609.30749, Sep 2026
- arXivImpossibleRubrics: Stress-Testing Generated Rubrics as Reward SignalsarXiv preprint arXiv:2609.16816, Sep 2026
- arXiv
- ACLLaoBench: A Large-Scale Multidimensional Lao Benchmark for Large Language ModelsIn Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul 2026
selected publications
- arXivFlagEval Findings Report: A Preliminary Evaluation of Large Reasoning Models on Automatically Verifiable Textual and Visual QuestionsarXiv preprint arXiv:2509.17177, Sep 2025
- arXivTowards analyzing and understanding the limitations of DPO: A theoretical perspectivearXiv preprint arXiv:2404.04626, 2024
- ACLBefore generation, align it! A novel and effective strategy for mitigating hallucinations in text-to-sql generationIn Association for Computational Linguistics (ACL Findings), 2024
- AAAIGraphix-t5: Mixing pre-trained transformers with graph-aware layers for text-to-sql parsingIn Proceedings of the AAAI Conference on Artificial Intelligence, 2023
- NeurIPSCan LLM already serve as a database interface? A big bench for large-scale database grounded text-to-sqlsIn Advances in Neural Information Processing Systems, 2023
- ACLS^2 SQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL ParsersIn Association for Computational Linguistics (ACL Findings), 2022
- COLINGSUN: Exploring intrinsic uncertainties in text-to-SQL parsersIn Proceedings of the 29th International Conference on Computational Linguistics, 2022
- SIGKDDProton: Probing schema linking information from pre-trained language models for text-to-sql parsingIn Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022
- arXivA survey on text-to-sql parsing: Concepts, methods, and future directionsarXiv preprint arXiv:2208.13629, 2022