





Welcome to the MURGe-Lab!
Our MURGe-Lab (Multimodal Understanding, Reasoning, and Generation for Language Lab; pronounced as "merge-lab") has research interests in statistical natural language processing and machine learning, with a focus on multimodal, grounded, and embodied semantics (i.e., language with vision and speech, for robotics), human-like language generation and Q&A/dialogue, and interpretable and structured deep learning. We are a group of PhD, MS, BS, and visiting students who work with Prof. Mohit Bansal and collaborators in the Computer Science department (lab located in Brooks Building FB-241C) at the University of North Carolina (UNC) Chapel Hill. We are also a part of the bigger UNC-NLP Group!
News
- Jun 2023 Two new Postdocs, Elias Stengel-Eskin and Jaehong Yoon will be joining our lab starting Fall 2023!
- May 2023 New papers in ACL 2023 and IJCAI 2023 and *SEM 2023 on faithful extractive summarization, meeting QA, single-frame bias in VidL, mixed fwd/rev cross-entropy, multimodal graph script induction, continual learning for code generation, exclusive supermasks for CL, modular multi-step reasoning, factuality of LLMs, sequential instruction understanding, and compositional LM differentiable prompting.
- Feb 2023 New papers in CVPR 2023 and ICLR 2023 and EACL 2023 and WACV 2023 on unified vision-text-layout DocAI, VLN with future-view semantics, hierarchical retrieval and step captioning, video-language pretraining, parameter-efficient audio-visual adapters, summarization programs, faithful decoding, edit-based prompt search, model belief graphs, social commonsense, multi-doc summarization + graph IE, long-distance video QA, perceiver-VL.
- Dec 2022 Congrats to Zineng Tang for being selected as winner of the CRA Outstanding Undergraduate Researcher Award 2023!
- Oct 2022 New papers in EMNLP 2022 and COLING 2022 and TACL on multimodal summarization + factuality, explanation hardness, mutual exclusivity for compositionality, action learning in interactive visual environments, LM bias, multimodal coreference in multi-turn dialogue, graph generative commonsense reasoning, and survey on data augmentation for limited data learning.
- Sept 2022 Congrats to Swarnadeep Saha for the Google NLP PhD Fellowship!
- Sept 2022 New papers in NeurIPS 2022 and ECCV 2022 on textless VL transformers, visual feature importance, ladder side-tuning, LLMs as fewshot video learners, T-few/IA3, gamified VL association benchmark, text to visual story generation, efficient video retrieval via audio replacement, etc.
- June 2022 New papers in NAACL 2022 on factual summarization pretraining, factuality evaluation, scene imagination commonsense, multilingual vision-lang-navigation, fine-grained CLIP captioning, masked POS modeling, curriculum learning, video intent discovery, multi-doc summarization clustering, interactive summarization, etc.
- Mar 2022 Congrats to Yichen Jiang for the Apple AI/ML PhD Fellowship!
- Dec 2021 Congrats to Shiyue Zhang for the Bloomberg PhD Fellowship!
- Nov 2021 Congrats to Ori+Ram and team on the CoNLL 2021 Best Paper Runner-Up Award!
- Sept 2021 New papers in NeurIPS 2021 on video-to-language knowledge distillation, query-based video highlight/saliency, socially-aligned feature importance explanations, video-lang understanding multi-task benchmark.
- Sept 2021 Congrats to Peter Hase for the Google PhD Fellowship! (article)
- Aug 2021 New papers in EMNLP 2021 on structure/compositionality, generalization, evaluation, and efficiency for explainability, commonsense, navigation, story visualization, and summarization.
- July 2021 Thanks to NSF for funding our ENGAGE NSF-AI Institute! (article)
- June 2021 Congrats to Jie and co-authors on the CVPR 2021 Best Student Paper Honorable Mention!
- May 2021 9 new papers in ICML 2021 and ACL 2021.
- Apr 2021 Congrats to Xiang and co-authors on the EACL 2021 Best Long Paper Award Honorable Mention!
- Apr 2021 9 new papers in NAACL 2021 (7 long, 1 short, 1 demo).
- Mar 2021 Congrats to Jie Lei for the Adobe Research Fellowship!
- Feb 2021 5 new papers in AAAI 2021, EACL 2021, and CVPR 2021.
- Dec 2020 Hao's work on vokenization covered in MIT Technology Review.
- Sept 2020 13 new papers in EMNLP 2020 (8 in main conference, 5 in findings).
- Aug 2020 Thanks to UNC for the UNC Phillip & Ruth Hettleman Prize for Artistic and Scholarly Achievement.
- May 2020 Thanks to IJCAI for the 'IJCAI Early Career Spotlight'.
- Apr 2020 8 new papers (6 in ACL 2020, 1 in IJCAI 2020, and 1 in ECCV 2020).
- Feb 2020 Thanks to Amazon for the Amazon Machine Learning Research Award (blog post).
- Jan 2020 Thanks to DARPA for the DARPA Director's Fellowship.
- Jan 2020 Thanks to Microsoft for the Microsoft Investigator Fellowship (MSR blog post).
- Dec 2019 Congrats to Sweta Karlekar and Han Guo for winning the Runner-Up and Finalist positions in the CRA Outstanding Undergraduate Researcher Awards! (link)
- Nov 2019 Three EMNLP workshops' keynote slides uploaded: MRQA, LANTERN, WNGT).
- Nov 2019 4 new papers in AAAI 2020.
- Aug 2019 Congrats to Peter Hase for the Royster Society PhD Fellowship!
- Aug 2019 5 new papers in EMNLP 2019.
- July 2019 Congrats to Hyounghun for ACL 2019 Best Short Paper Nomination!
- July 2019 We have a Postdoc opening - please apply!
- July 2019 Thanks for the NSF-CAREER Award (details).
- July 2019 Thanks for the Google Focused Research Award (details).
- June 2019 Congrats to Hao Tan for the Bloomberg Data Science Ph.D. Fellowship!
- May 2019 6 new papers in ACL 2019.
- Apr 2019 Congrats to Darryl Hannan for the 3-year NSF PhD Fellowship!
- Mar 2019 Congrats to Hao Tan for 1st Rank on the Room-to-Room Vision-Language-Navigation Leaderboard!
- Feb 2019 5 new papers: 3 in NAACL 2019, 1 in CVPR 2019, 1 in ICRA 2019.
- Jan 2019. Congrats to Ramakanth Pasunuru for being awarded the 2-year Microsoft Research PhD Fellowship!
- Dec 2018. Congrats to Han Guo for being awarded the CRA Outstanding Undergraduate Researcher Award Honorable Mention!
- Nov 2018. Thanks for the Salesforce Research Deep Learning Grant!
- Oct 2018. 2 new papers in AAAI 2019 (16% acceptance rate).
- Sept 2018. Thanks to Facebook and IBM for the faculty research awards!
- Aug 2018. 7 new papers (6 in EMNLP; 1 in CoNLL).
- July 2018. Thanks to Army Research Office for the ARO Young Investigator Program (YIP) Award.
-
July 2018.
1st rank in EMNLP FEVER (Fact Extraction & VERification) shared task (congrats Yixin and Haonan)! [Press Article] - June 2018. COLING paper on dynamic-MTL selected as "Area Chair Favorites" (congrats Han+Ram)!
- Apr 2018. 4 new papers (2 in ACL; 1 in TACL; 1 in WiNLP).
- Apr 2018. Congrats to Lisa Bauer for being awarded the 3-year NSF PhD Fellowship!
- Mar 2018. Thanks to Adobe for the Adobe Research Award.
- Feb 2018. 9 new 2018 papers in NAACL, CVPR, AAAI, WACV.
- Sept 2017. Thanks to DARPA for the DARPA Young Faculty Award (link).
- Sept 2017. Thanks to Facebook for the Facebook ParlAI Research Award.
- July 2017. 3 papers at EMNLP 2017 and 2 papers at the Summarization-Frontiers and RepEval workshops.
- June 2017. Top single model results on the RepEval-NLI Shared Task at EMNLP 2017 (congrats Yixin!).
- June 2017. Outstanding Paper Award at ACL 2017 (congrats Ram!).
- Feb 2017. Thanks to Google for a Google Faculty Research Award (link).
- Nov 2016. 3 papers on navigational instruction generation, coherent dialogue w/ attn-LMs, and on context-RNN-GAN models to appear at AAAI 2017 and HRI 2017.
- July 2016. 5 papers to appear at EMNLP 2016: visual story sorting, visual question relevance, neural network interpretation (for politeness), machine comprehension/Q&A, and character-based sentence embedding models.
- July 2016. Best paper award at ACL 2016 Repl4NLP workshop for paper on mapping unseen words, and paper on end-to-end joint neural models for entity+relation extraction to appear at ACL 2016.
- Mar 2016. Thanks to Bloomberg for a Bloomberg Data Science Research Grant (link).
- Feb 2016. Paper on universal sentence embeddings selected as an oral at ICLR 2016.
- Jan 2016. Our work on AI for computational humor was covered in MIT Technology Review and Newsweek.
- Nov 2015. Nvidia paper award at NIPS 2015 Multimodal ML workshop for paper on navigational instruction following.
- Dec 2014. Thanks for an IBM Faculty Award and a Google Faculty Research Award (link).