Good Papers
In-context learning & promptingBeijing JiaotongAlibaba DAMO AcademyAlibaba Group

ChatIE: Zero-Shot Information Extraction via Chatting with ChatGPT

ChatIE reframes zero-shot information extraction as multi-turn ChatGPT dialogue, surpassing some fully supervised models on several benchmark datasets.

Wei, Xiang, Xingyu Cui, Ning Cheng, Xiaobin Wang, Xin Zhang, Shen Huang, Pengjun Xie, Jinan Xu, Yufeng Chen, Meishan Zhang, Yong Jiang, Wenjuan Han

Published Feb 20, 2023147 citationsarXiv ↗

76%
OverallHighly rated
?
OverallHighly ratedVote to see the scoreThe exact score shows once you've voted, so every vote is your own call. The first half of each home page shelf shows its scores.
Readers
–

Only vote on papers you've read. Sign in with GitHub to vote.

AI panel10/21reviewers recommend it
lenient 5/5
medium 5/11
strict 0/5
AI panel?Vote to see what the 20 AI reviewers said
Panel consensus
ChatIE impressively turns zero-shot information extraction into multi-turn ChatGPT dialogue and surpasses full-shot models on benchmarks, but its multi-turn design hides heavy inference costs, lacks open-source implementation, and never proves its gains survive beyond ChatGPT or…

Abstract

Zero-shot information extraction (IE) aims to build IE systems from the unannotated text. It is challenging due to involving little human intervention. Challenging but worthwhile, zero-shot IE reduces the time and effort that data labeling takes. Recent efforts on large language models (LLMs, e.g., GPT-3, ChatGPT) show promising performance on zero-shot settings, thus inspiring us to explore prompt-based methods. In this work, we ask whether strong IE models can be constructed by directly prompting LLMs. Specifically, we transform the zero-shot IE task into a multi-turn question-answering problem with a two-stage framework (ChatIE). With the power of ChatGPT, we extensively evaluate our framework on three IE tasks: entity-relation triple extract, named entity recognition, and event extraction. Empirical results on six datasets across two languages show that ChatIE achieves impressive performance and even surpasses some full-shot models on several datasets (e.g., NYT11-HRL). We believe that our work could shed light on building IE models with limited resources.