Xiujun Li
Multimodal LLMs, LLMs, NLP, Vision and Language
I am currently a Staff Research Scientist at Apple. I got my PhD from UW CSE with Yejin Choi on 2024. Before I worked five years at Microsoft Research. My research experience is on Dialog, Deep Reinforcement Learning, NLP, Vision and Language, Multimodal LLMs etc. Recently, I am also interested in Video Generation.
Selected Papers Google Scholar
Multimodal LLMs
-
Xiujun Li*, Yujie Lu*, Zhe Gan, Jianfeng Gao, William Yang Wang, Yejin Choi
Text as Images: Can Multimodal Large Language Models Follow Printed Instructions in Pixels?, arXiv 2024
-
Yujie Lu, Xiujun Li, Tsu-Jui Fu, Miguel Eckstein, William Yang Wang
From Text to Pixel: Advancing Long-Context Understanding in MLLMs, arXiv 2024
-
Zhangheng Li, Keen You, Haotian Zhang, Di Feng, Harsh Agrawal, Xiujun Li, Mohana Prasad Sathya Moorthy, Jeff Nichols, Yinfei Yang, Zhe Gan
Ferret-UI 2: Mastering Universal User Interface Understanding Across Platforms, arXiv 2024
Vision and Language
-
Pengchuan Zhang*, Xiujun Li*, Xiaowei Hu, Jianwei Yang, Lei Zhang, Lijuan Wang, Yejin Choi, Jianfeng Gao
VinVL: Making Visual Representations Matter in Vision-Language Models, CVPR 2021
-
Xiujun Li, Xi Yin, Chunyuan Li, Xiaowei Hu, Pengchuan Zhang, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, Jianfeng Gao
Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks, ECCV 2020
-
Xiujun Li, Chunyuan Li, Qiaolin Xia, Yonatan Bisk, Asli Celikyilmaz, Jianfeng Gao, Noah Smith, Yejin Choi
Robust navigation with language pretraining and stochastic sampling, EMNLP 2019
Dialog, RL
-
Xiujun Li, Yun-Nung Chen, Lihong Li, Jianfeng Gao, Asli Celikyilmaz
End-to-End Task-Completion Neural Dialogue Systems, IJCNLP 2017 -
Baolin Peng, Xiujun Li, Lihong Li, Jianfeng Gao, Asli Celikyilmaz, Sungjin Lee, Kam-Fai Wong
Composite Task-Completion Dialogue System via Hierarchical Deep Reinforcement Learning, EMNLP 2017 -
Baolin Peng, Xiujun Li, Jianfeng Gao, Jingjing Liu, Kam-Fai Wong and Shang-Yu Su
Deep Dyna-Q: Integrating Planning for Task-Completion Dialogue Policy Learning, ACL 2018