(*- equal contributions)
all publications
(You can also checkout my google scholar / semantic scholar / acl anthology pages.)
Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context
Gemini Team, Google
Technical Report’24
Gemini: A Family of Highly Capable Multimodal Models
Gemini Team, Google
Technical Report’23
DePlot: One-shot visual language reasoning by plot-to-table translation
Fangyu Liu*, Julian Martin Eisenschlos*, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Wenhu Chen, Nigel Collier, Yasemin Altun
ACL’23-Findings (Findings of the Association for Computational Linguistics: ACL 2023), July 2023
[code/models (t5x)] [code/models (huggingface)] [demo] [LlamaIndex]
MatCha: Enhancing Visual Language Pretraining with Math Reasoning and Chart Derendering
Fangyu Liu, Francesco Piccinno, Syrine Krichene, Chenxi Pang, Kenton Lee, Mandar Joshi, Yasemin Altun, Nigel Collier, Julian Martin Eisenschlos
ACL’23 (The 61st Annual Meeting of the Association for Computational Linguistics), July 2023
[code/models (t5x)] [code/models (huggingface)] [qa-demo]
Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding
Kenton Lee*, Mandar Joshi*, Iulia Turc, Hexiang Hu, Fangyu Liu, Julian Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, Kristina Toutanova
ICML’23 (The 40th International Conference on Machine Learning), July 2023
[code/models (t5x)] [code/models (huggingface)]
Compositional Zero-Shot Domain Transfer with Text-to-Text Models
Fangyu Liu, Qianchu Liu, Shruthi Bannur, Fernando Pérez-García, Naoto Usuyama, Sheng Zhang, Tristan Naumann, Aditya Nori, Hoifung Poon, Javier Alvarez-Valle, Ozan Oktay, Stephanie L. Hyland
TACL’23 (Transactions of the Association for Computational Linguistics), 2023
Visual Spatial Reasoning
Fangyu Liu, Guy Emerson, Nigel Collier
TACL’23 (Transactions of the Association for Computational Linguistics), 2023
[code/data]
WinoDict: Probing language models for in-context word acquisition
Julian Martin Eisenschlos, Jeremy R. Cole, Fangyu Liu, William W. Cohen
EACL’23 (The 17th Conference of the European Chapter of the Association for Computational Linguistics), May 2023
Best Paper Award
[code/data]
Exposing Cross-Lingual Lexical Knowledge from Multilingual Sentence Encoders
Ivan Vulić, Goran Glavaš, Fangyu Liu, Nigel Collier, Edoardo Maria Ponti, Anna Korhonen
EACL’23 (The 17th Conference of the European Chapter of the Association for Computational Linguistics), May 2023
On Reality and the Limits of Language Data: Aligning LLMs with Human Norms
Nigel Collier, Fangyu Liu, Ehsan Shareghi
CogSci’23 (The 45th Annual Meeting of the Cognitive Science Society), July 2023
[data]
Revisiting Parameter-Efficient Tuning: Are We Really There Yet?
Guanzheng Chen, Fangyu Liu, Zaiqiao Meng, Shangsong Liang
EMNLP’22 (The 2022 Conference on Empirical Methods in Natural Language Processing), December 2022
[code]
Sharpness-Aware Minimization with Dynamic Reweighting
Wenxuan Zhou, Fangyu Liu, Huan Zhang, Muhao Chen
EMNLP’22-Findings (Findings of the Association for Computational Linguistics: EMNLP 2022), December 2022
Improving Bilingual Lexicon Induction with Cross-Encoder Reranking
Yaoyiran Li, Fangyu Liu, Ivan Vulić, Anna Korhonen
EMNLP’22-Findings (Findings of the Association for Computational Linguistics: EMNLP 2022), December 2022
[code]
TweetNLP: Cutting-Edge Natural Language Processing for Social Media
Jose Camacho-Collados, Kiamehr Rezaee, Talayeh Riahi, Asahi Ushio, Daniel Loureiro, Dimosthenis Antypas, Joanne Boisson, Luis Espinosa-Anke, Fangyu Liu, Eugenio Martínez-Cámara, Gonzalo Medina, Thomas Buhrmann, Leonardo Neves, Francesco Barbieri
EMNLP’22 (demo track) (The 2022 Conference on Empirical Methods in Natural Language Processing: System Demonstrations), December 2022
[website]
Do ever larger octopi still amplify reporting biases? Evidence from judgments of typical colour
Fangyu Liu, Julian Martin Eisenschlos, Jeremy R. Cole, Nigel Collier
AACL-IJCNLP’22 (The 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing), November 2022
[poster]
How to tackle an emerging topic? Combining strong and weak labels for Covid news NER
Aleksander Ficek, Fangyu Liu, Nigel Collier
AACL-IJCNLP’22 (The 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing), November 2022
[code]
IGLUE: A Benchmark for Transfer Learning across Modalities, Tasks, and Languages
Emanuele Bugliarello, Fangyu Liu, Jonas Pfeiffer, Siva Reddy, Desmond Elliott, Edoardo Maria Ponti, and Ivan Vulić
ICML’22 (The 39th International Conference on Machine Learning), July 2022
[code] [website]
TaCL: Improving BERT Pre-training with Token-aware Contrastive Learning
Yixuan Su, Fangyu Liu, Zaiqiao Meng, Lei Shu, Ehsan Shareghi, Nigel Collier
NAACL’22-Findings (Findings of the Association for Computational Linguistics: NAACL 2022), July 2022
[code] [huggingface models]
Language Models Can See: Plugging Visual Controls in Text Generation
Yixuan Su, Tian Lan*, Yahui Liu*, Fangyu Liu*, Dani Yogatama, Yan Wang, Lingpeng Kong, Nigel Collier
arxiv preprint, May 2022
[code]
Prix-LM: Pretraining for Multilingual Knowledge Base Construction
Wenxuan Zhou*, Fangyu Liu*, Ivan Vulić, Nigel Collier, Muhao Chen
ACL’22 (The 60th Annual Meeting of the Association for Computational Linguistics), May 2022
[code] [huggingface model]
Rewire-then-Probe: A Contrastive Recipe for Probing Biomedical Knowledge of Pretrained Language Models
Zaiqiao Meng*, Fangyu Liu*, Ehsan Shareghi, Yixuan Su, Charlotte Collins, Nigel Collier
ACL’22 (The 60th Annual Meeting of the Association for Computational Linguistics), May 2022
[code]
Improving Word Translation via Two-Stage Contrastive Learning
Yaoyiran Li, Fangyu Liu, Nigel Collier, Anna Korhonen, Ivan Vulić
ACL’22 (The 60th Annual Meeting of the Association for Computational Linguistics), May 2022
[code]
Fine-Grained Controllable Text Generation Using Non-Residual Prompting
Fredrik Carlsson, Joey Öhman, Fangyu Liu, Severine Verlinden, Joakim Nivre, Magnus Sahlgren
ACL’22 (The 60th Annual Meeting of the Association for Computational Linguistics ), May 2022
[code]
Trans-Encoder: Unsupervised sentence-pair modelling through self- and mutual-distillations
Fangyu Liu, Yunlong Jiao, Jordan Massiah, Emine Yilmaz, Serhii Havrylov
ICLR’22 (The 10th International Conference on Learning Representations), April 2022
Also presented at NeurIPS 2021 Workshop: Self-Supervised Learning - Theory and Practice, December 2021
[code] [huggingface models] [talk] [amazon.science blog post]
MirrorWiC: On Eliciting Word-in-Context Representations from Pretrained Language Models
Qianchu Liu*, Fangyu Liu*, Nigel Collier, Anna Korhonen, Ivan Vulić
CoNLL’21 (The 25th Conference on Computational Natural Language Learning), November 2021
[code] [talk]
Visually Grounded Reasoning across Languages and Cultures
Fangyu Liu*, Emanuele Bugliarello*, Edoardo Maria Ponti, Siva Reddy, Nigel Collier, Desmond Elliott
EMNLP’21 (The 2021 Conference on Empirical Methods in Natural Language Processing), November 2021
Best Long Paper Award
[website] [code] [talk] [poster]
Fast, Effective, and Self-Supervised: Transforming Masked Language Models into Universal Lexical and Sentence Encoders
Fangyu Liu, Ivan Vulić, Anna Korhonen, Nigel Collier
EMNLP’21 (The 2021 Conference on Empirical Methods in Natural Language Processing), November 2021
[code] [huggingface models] [talk] [poster]
Contrastive Out-of-Distribution Detection for Pretrained Transformers
Wenxuan Zhou, Fangyu Liu, Muhao Chen
EMNLP’21 (The 2021 Conference on Empirical Methods in Natural Language Processing), November 2021
[code]
Mixture-of-Partitions: Infusing Large Biomedical Knowledge Graphs into BERT
Zaiqiao Meng, Fangyu Liu, Thomas Clark, Ehsan Shareghi, Nigel Collier
EMNLP’21 (The 2021 Conference on Empirical Methods in Natural Language Processing), November 2021
[code] [talk]
Learning Domain-Specialised Representations for Cross-Lingual Biomedical Entity Linking
Fangyu Liu, Ivan Vulić, Anna Korhonen, Nigel Collier
ACL-IJCNLP’21 (The Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing), August 2021
[code/data] [huggingface models] [talk]
Self-Alignment Pretraining for Biomedical Entity Representations
Fangyu Liu, Ehsan Shareghi, Zaiqiao Meng, Marco Basaldella, Nigel Collier
NAACL’21 (2021 Annual Conference of the North American Chapter of the Association for Computational Linguistics), June 2021
Also presented at NeurIPS 2020 Workshop: Self-Supervised Learning - Theory and Practice, December 2020
[code] [huggingface models] [slides] [poster] [talk] [bibtex]
(check out an implementation and tutorial by NVIDIA’s NeMo team)
Visual Pivoting for (Unsupervised) Entity Alignment
Fangyu Liu, Muhao Chen, Dan Roth, Nigel Collier
AAAI’21 (The 35th AAAI Conference on Artificial Intelligence), February 2021
[code] [slides]
COMETA: A Corpus for Medical Entity Linking in the Social Media
Marco Basaldella*, Fangyu Liu*, Ehsan Shareghi, Nigel Collier
EMNLP’20 (The 2020 Conference on Empirical Methods in Natural Language Processing), November 2020
[website] [code] [huggingface-bioreddit-bert] [talk] [bibtex]
Upgrading the Newsroom: An Automated Image Selection System for News Articles
Fangyu Liu, Rémi Lebret, Didier Orel, Philippe Sordet, Karl Aberer
ACM TOMM (ACM Transactions on Multimedia Computing Communications and Applications), July 2020
[demo] [slides] [bibtex]
HAL: Improved Text-Image Matching by Mitigating Visual Semantic Hubs
Fangyu Liu*, Rongtian Ye*, Xun Wang*, Shuaipeng Li
AAAI’20 (The 34th AAAI Conference on Artificial Intelligence), February 2020
[code] [poster] [bibtex]
A Strong and Robust Baseline for Text-Image Matching
Fangyu Liu, Rongtian Ye
ACL’19 SRW (ACL 2019 Student Research Workshop), August 2019
Visually Grounded Cross-Lingual Transfer Learning
Fangyu Liu, Rémi Lebret, Karl Aberer
NAACL 2019 Workshop on Shortcomings in Vision and Language, June 2019
[poster]
A Novel Hybrid Machine Learning Model for Auto-Classification of Retinal Diseases
C.-H. Huck Yang, Jia-Hong Huang, Fangyu Liu, Fang-Yi Chiu, Mengya Gao, Weifeng Lyu, I-Hung Lin, Jesper Tegner
ICML 2018 Workshop on Computational Biology, July 2018
3DCNN-DQN-RNN: A Deep Reinforcement Learning Framework for Semantic Parsing of Large-scale 3D Point Clouds
Fangyu Liu*, Shuaipeng Li*, Liqiang Zhang, Chenghu Zhou, Rongtian Ye, Yuebin Wang, Jiwen Lu
ICCV’17 (The 2017 IEEE International Conference on Computer Vision), October 2017
[poster] [slides] [bibtex]