Bowen Qin

prof_pic.jpg

National University of Singapore (NUS), Singapore. eyuansu71@gmail.com

I am a 1st-year Ph.D. student at the National University of Singapore (NUS), advised by Prof. Yao Lu.

Previously, I was a researcher at the Beijing Academy of Artificial Intelligence (BAAI), specializing in the evaluation, alignment, and code intelligence of large language models (LLMs). Previously, I obtained my master’s degree with top honors from the Shenzhen Institute of Advanced Technology (SIAT), Chinese Academy of Sciences (CAS), under the guidance of Prof. Min Yang in 2023.

news

Sep 17, 2026 We introduced ImpossibleRubrics, a benchmark for testing whether generated rubrics reward unsupported answers over honest responses to impossible tasks. Announcement.
Aug 21, 2026 I shared an analysis of context compaction across roughly 400,000 coding-agent turns, examining tool errors and tool use around compaction. Discussion.
May 23, 2026 I introduced LogDx-CI, a benchmark comparing log-reduction tools for LLM diagnosis of real CI failures. Announcement.
May 13, 2026 We introduced Agent Retrieval Bench to evaluate the retrieval of task-relevant repository files for coding agents. Announcement.

recent publications

  1. arXiv
    BIRD-RL: Scaling Agentic Reinforcement Learning over Stateful Data-Centric Environments
    Ge Qu, Jinyang Li, Xiaohan Xu, and 6 more authors
    arXiv preprint arXiv:2610.29426, Oct 2026
  2. arXiv
    ORCA: Evaluating LLMs on Data Science Code Translation
    Xiaolong Li, Jinyang Li, Bowen Qin, and 5 more authors
    arXiv preprint arXiv:2609.30749, Sep 2026
  3. arXiv
    ImpossibleRubrics: Stress-Testing Generated Rubrics as Reward Signals
    Bowen Qin, Yi Xie, Yesheng Liu, and 1 more author
    arXiv preprint arXiv:2609.16816, Sep 2026
  4. arXiv
    MoM: Memory of Memory
    Bowen Qin and Yao Lu
    arXiv preprint arXiv:2609.25054, Sep 2026
  5. ACL
    LaoBench: A Large-Scale Multidimensional Lao Benchmark for Large Language Models
    Jian Gao, Richeng Xuan, Zhaolu Kang, and 9 more authors
    In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Jul 2026

selected publications

  1. arXiv
    FlagEval Findings Report: A Preliminary Evaluation of Large Reasoning Models on Automatically Verifiable Textual and Visual Questions
    Bowen Qin, C Yue, F Yin, and 7 more authors
    arXiv preprint arXiv:2509.17177, Sep 2025
  2. arXiv
    Towards analyzing and understanding the limitations of DPO: A theoretical perspective
    Duanyu Feng, Bowen Qin, Chen Huang, and 2 more authors
    arXiv preprint arXiv:2404.04626, 2024
  3. ACL
    Before generation, align it! A novel and effective strategy for mitigating hallucinations in text-to-sql generation
    Ge Qu, Jinyang Li, Bowen Li, and 4 more authors
    In Association for Computational Linguistics (ACL Findings), 2024
  4. AAAI
    Graphix-t5: Mixing pre-trained transformers with graph-aware layers for text-to-sql parsing
    Jinyang Li, Binyuan Hui, Reynold Cheng, and 7 more authors
    In Proceedings of the AAAI Conference on Artificial Intelligence, 2023
  5. NeurIPS
    Can LLM already serve as a database interface? A big bench for large-scale database grounded text-to-sqls
    Jinyang Li, Binyuan Hui, Ge Qu, and 8 more authors
    In Advances in Neural Information Processing Systems, 2023
  6. ACL
    S^2 SQL: Injecting Syntax to Question-Schema Interaction Graph Encoder for Text-to-SQL Parsers
    Binyuan Hui, Ruiying Geng, Lihan Wang, and 4 more authors
    In Association for Computational Linguistics (ACL Findings), 2022
  7. COLING
    SUN: Exploring intrinsic uncertainties in text-to-SQL parsers
    Bowen Qin, Lihan Wang, Binyuan Hui, and 7 more authors
    In Proceedings of the 29th International Conference on Computational Linguistics, 2022
  8. SIGKDD
    Proton: Probing schema linking information from pre-trained language models for text-to-sql parsing
    Lihan Wang, Bowen Qin, Binyuan Hui, and 8 more authors
    In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 2022
  9. arXiv
    A survey on text-to-sql parsing: Concepts, methods, and future directions
    Bowen Qin, Binyuan Hui, Lihan Wang, and 8 more authors
    arXiv preprint arXiv:2208.13629, 2022