news

Sep 17, 2026 We introduced ImpossibleRubrics, a benchmark for testing whether generated rubrics reward unsupported answers over honest responses to impossible tasks. Announcement.
Aug 21, 2026 I shared an analysis of context compaction across roughly 400,000 coding-agent turns, examining tool errors and tool use around compaction. Discussion.
May 23, 2026 I introduced LogDx-CI, a benchmark comparing log-reduction tools for LLM diagnosis of real CI failures. Announcement.
May 13, 2026 We introduced Agent Retrieval Bench to evaluate the retrieval of task-relevant repository files for coding agents. Announcement.