Conformal Prediction for Natural Language Processing: A Survey
Campos, M.
;
Farinhas, A. F.
;
Zerva, C
;
Figueiredo, M. A. T.
;
Martins, A.
Transactions of the Association for Computational Linguistics Vol. 12, Nº , pp. 1497 - 1516, July, 2024.
ISSN (print):
ISSN (online): 2307-387X
Scimago Journal Ranking: 4,01 (in 2023)
Digital Object Identifier: 10.1162/tacl_a_00715
Abstract
The rapid proliferation of large language models and natural language processing (NLP) applications creates a crucial need for uncertainty quantification to mitigate risks such as Hallucinations and to enhance decision-making reliability in critical applications. Conformal prediction is emerging as a theoretically sound and practically useful framework, combining flexibility with strong statistical guarantees. Its model-agnostic and distribution-free nature makes it particularly promising to address the current shortcomings of NLP systems that stem from the absence of uncertainty quantification. This paper provides a comprehensive survey of conformal prediction techniques, their guarantees, and existing applications in NLP, pointing to directions for future research and open challenges.
