Qi Ji, a doctoral student from the Knowledge Engineering Group of the Department of Computer Science and Technology at Tsinghua University, has been awarded the Outstanding Paper Award at the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP 2023). The conference, which is a leading international event in the field of artificial intelligence and natural language processing, was recently held in Singapore.

Qi Ji’s paper, titled “Preserving Knowledge Invariance: Rethinking Robustness Evaluation of Open Information Extraction,” was recognized for its innovative approach to the robustness of natural language processing (NLP) models. The paper was supervised by Professor Xu Bin and co-supervised by Professor Li Juanzi and Assistant Professor Hou Lei, all from the Knowledge Engineering Group of the Department of Computer Science and Technology at Tsinghua University. Doctoral students Wang Xiaozhi, Yu Jifan, Zeng Kaisheng and Liu Jinxin contributed to the award-winning paper.

The paper presents the first benchmark that simulates the evaluation of open information extraction models in the real world, where the syntactic and expressive distributions under the same knowledge meaning may drift variously. The authors designed and annotated a large-scale testbed in which each example is a knowledge-invariant clique that consists of sentences with structured knowledge of the same meaning but with different syntactic and expressive forms. Experiments on typical models in the past decade and a representative large language model showed that, the proposed benchmark could effectively evaluate the accuracy and robustness of open information extraction models in real-world scenarios.

EMNLP is hosted annually by the Special Interest Group on Linguistic Data & Corpus-based Approaches to Natural Language Processing (SIGDAT) under the Association for Computational Linguistics (ACL).

Editor: Li Han