Hi! 👋

I am a fourth year computer science PhD student at Vislang Lab at Rice University, advised by Prof. Vicente Ordonez.

My research focuses on developing general intelligence by integrating visual and linguistic information to enhance world understanding. I am particularly interested in AI Agents and multimodal reasoning.

I completed my MS in Computer Science from Columbia University, where I worked on multimodal emotion detection in Speech Lab, advised by Prof. Julia Hirschberg. I have also worked with Prof. Shih-Fu Chang at Columbia University.

Previously, I graduated from Ewha Womans University with B.S. in Computer Science and Engineering, and Scranton Honors Program , where I was advised by Prof. Dongbo Min and by Prof. Hyun-Seok Park .

🔥 News

• [08/2026] Our paper HypoExplore is accepted to EMNLP 2026 Findings and “Beyond Referring Expressions: Scenario Comprehension Visual Grounding” is accepted to EMNLP 2026 Main. 🎉
• [07/2026] Our paper “SoftmaxGRPO: Learning to Reason using Softmax Advantage Group Estimation” is accepted to COLM 2026. 🎉
• [05/2026] I joined SONY AI as a research intern this summer.
• [01/2026] Our paper “ProxyThinker: Test-Time Guidance through Small Visual Reasoners” is accepted to ICLR 2026. 🎉
• [09/2025] Our paper “Evaluating Text-to-Image and Text-to-Video Synthesis with a Conditional Frechet Distance” is accepted to WACV 2026. 🎉
• [09/2024] Our paper “PropTest: Automatic Property Testing for Improved Visual Programming” is accepted to EMNLP 2024 Findings. 🎉
• [03/2024] Our paper “Multi-Modality Multi-Loss Fusion Network” is accepted to NAACL 2024 (Oral). 🎉

📝 Publications

Preprints

Publications

Agentic Discovery with Active Hypothesis Exploration for Visual Recognition
Jaywon Koo, Jefferson Hernandez, Ruozhen He, Hanjie Chen, Chen Wei, Vicente Ordonez
The 2026 Conference on Empirical Methods in Natural Language Processing, Findings. EMNLP Findings, 2026.
Paper | Code

Beyond Referring Expressions: Scenario Comprehension Visual Grounding
Ruozhen He, Nisarg A Shah, Qihua Dong, Zilin Xiao, Jaywon Koo, Vicente Ordonez
The 2026 Conference on Empirical Methods in Natural Language Processing. EMNLP 2026.
Paper | Project Page

SoftmaxGRPO: Learning to Reason using Softmax Advantage Group Estimation
Jefferson Hernandez, Jaywon Koo, Zilin Xiao, Chen Wei, Vicente Ordonez
Conference on Language Modeling. COLM 2026.
Paper

ProxyThinker: Test-Time Guidance through Small Visual Reasoners
Zilin Xiao, Jaywon Koo, Siru Ouyang, Jefferson Hernandez, Yu Meng, Vicente Ordonez
The Fourteenth International Conference on Learning Representations. ICLR 2026.
Paper | Code

Evaluating Text-to-Image and Text-to-Video Synthesis with a Conditional Frechet Distance
Jaywon Koo*, Jefferson Hernandez*, Moayed Haji-Ali, Ziyan Yang, Vicente Ordonez (*Joint First Authors)
The IEEE/CVF Winter Conference on Applications of Computer Vision. WACV 2026.
Paper | Code

PropTest: Automatic Property Testing for Improved Visual Programming
Jaywon Koo, Ziyan Yang, Paola Cascante-Bonilla, Baishakhi Ray, Vicente Ordonez
Findings of Empirical Methods in Natural Language Processing. Findings of EMNLP 2024.
Paper | Project Page

Multimodal Multi-loss Fusion Network for Sentiment Analysis
Zehui Wu*, Ziwei Gong*, Jaywon Koo, Julia Hirschberg (*Joint First Authors)
Annual Conference of the North American Chapter of the Association for Computational Linguistics. NAACL 2024 (Oral).
Paper | Code

Beyond Grounding: Extracting Fine-Grained Event Hierarchies Across Modalities
Hammad Ayyubi, Christopher Thomas, Lovish Chum, Rahul Lokesh, Long Chen, Yulei Niu, Xudong Lin, Xuande Feng, Jaywon Koo, Sounak Ray, Shih-Fu Chang
Proceedings of the AAAI Conference on Artificial Intelligence. AAAI-24.
Paper