Iowa State University Computer Science professors Qi Li, Yang Li, and Wallapak Tavanapong, and graduate students, will present four research articles at the 63rd Annual Meeting of the Association for Computational Linguistics (ACL) from July 27th to August 1st. ACL is a leading conference in the field of computational linguistics and natural language processing.
The work of Assistant Professor Qi Li and graduate student Qing Wang in the paper “Towards a More Generalized Approach in Open Relation Extraction” presents MixORE, a two-phase framework for a more generalized OpenRE setting. Unlike prior methods, MixORE models unlabeled data as a mixture of known and novel instances, combining novel relation detection with open-world semi-supervised joint learning. This improves adaptability to real-world applications and addresses key limitations of existing OpenRE approaches.
Another paper by Dr. Qi Li, written with graduate student Yuepei Li, is titled “Investigating Context Faithfulness in Large Language Models: The Role of Memory Strength and Evidence Style.” In this paper, they explore how receptive language models (LLMs) are to external evidence under knowledge conflict scenarios. The study examines the impact of memory strength and evidence style on LLMs’ receptiveness. The show showed that LLMs are less receptive to external evidence when memory strength is high, while presenting paraphrased evidence significantly improves their receptiveness compared to simple repetition or adding details.
In their paper "AutoMixer: Checkpoint Artifacts as Automatic Data Mixers," Assistant Professor Yang Li, Dr. Ernie Chang of Meta, and other Meta collaborators introduce AutoMixer—a novel data selection strategy to improve LLM pretraining across diverse tasks. While language models benefit from acquiring a range of capabilities, identifying the right data mixtures remains difficult due to the complex relationship between data and tasks. The authors observe that checkpoint models develop emerging capabilities at different stages of training. Building on this insight, AutoMixer selects checkpoint artifacts based on their benchmark performance and uses their aggregated first-order influence over source data to guide data mixing. Extensive experiments show that AutoMixer significantly boosts pretraining performance across a wide range of benchmarks.
The final research article presented is by Assistant Professor Qi Li and Professor Wallapak Tavanapong, alongside graduate students Seok Hwan Song and Mohna Chakraborty. The paper “Is Large Language Model Performance on Reasoning Tasks Impacted by Different Ways Questions Are Asked?”, examines how question types affect LLM reasoning performance. Using quantitative and deductive reasoning datasets, the authors compare short answer (SAQ), multiple-choice (MCQ), and true/false (TFQ) types. The study is motivated by the observation that many researchers evaluate models using multiple benchmarks with different question types without considering how these differing formats may impact the LLM performance. These findings offer important guidance for designing more consistent and fairer LLM benchmarks.