ICML 2026
Oral
Position: There are futures that benchmark-driven AI cannot see
Sobhan Lotfi, Ava Iranmanesh, Lachin Naghashyar, Ali Shirali, Fateme Nateghi Haredasht, Sanmi Koyejo, Philip Torr, Yong Suk Lee, Fazl Barez, Joel Lehman, Peter Norvig, Arvind Narayanan
Benchmarks decide which problems count as progress, and in doing so they quietly fix the set of futures a research culture can even aim at. We argue that the resulting blind spots are structural rather than incidental, and that they will not be fixed by adding more benchmarks.
ICML PDF
NeurIPS 2026
Datasets & Benchmarks
Under review
HearSayBench: Can LLMs Navigate from Abstract Human Rights to Lived Lives?
Sobhan Lotfi*, Ava Iranmanesh*, Ali Iranmanesh*, Liwei Jiang
A benchmark probing whether models recognize the unstated structural constraints that make standard advice impossible for people in adversarial conditions. Across 11 frontier and open-weight models we find a systemic 37% drop between understanding a situation and giving advice that respects it. Oral at the Pluralistic Alignment Workshop, ICML 2026.
Project page
IJHCS 2026
Vol. 209, 103737
Design Patterns of Human-AI Interfaces in Healthcare
Rui Sheng, Chuhan Shi, Sobhan Lotfi, Shiyi Liu, Adam Perer, Huamin Qu, Furui Cheng
A review of clinical human-AI interfaces yielding 15 recurring information entities and 12 design patterns, each grounded in interviews with 12 healthcare professionals and evaluated with 14 UI designers.
arXiv DOI
Preprint
Dynamic Implicit Neural Representations for Image Compression
Sobhan Lotfi, Peyman Paknezhad, Pooya Ashtari, Fateme Nateghi Haredasht, Yixing Jiang, Aleksandra Pizurica
Implicit neural representations with input-dependent dynamics for image compression.