Beyond Relevance: Utility-Centric Retrieval in the LLM Era

About This Tutorial

Retrieval-Augmented Generation (RAG) has become a foundational paradigm for enhancing large language models (LLMs) with external knowledge. While traditional information retrieval (IR) systems primarily optimize for relevance — matching queries to documents based on topical similarity — the emergence of LLMs has fundamentally reshaped what makes retrieved information useful.

This tutorial explores a paradigm shift from relevance-centric to utility-centric retrieval, where the value of retrieved information is measured not by its topical match to a query, but by its actual contribution to downstream LLM tasks. We cover:

• Section 1: Introduction
• Section 2: LLM-centric Utility in RAG
• Section 3: Utility Modeling and Optimization Methods
• Section 4: LLM Information Needs and Agentic RAG
• Section 5: Conclusions and Future Directions

This tutorial aims to generate broader attention to utility-centric retrieval issues in the LLM era, facilitate an understanding of the relevant literature, and lower the barrier to entry for interested researchers and practitioners.

Organizers

Hengran Zhang

ICT, CAS

Minghao Tang

ICT, CAS

Keping Bi

ICT, CAS

Jiafeng Guo

ICT, CAS

Schedule

Format: Half-day (3.5-hour) lecture-style tutorial. Monday, 20 July 2026.

BibTeX

@article{zhang2026beyond,
  title={Beyond Relevance: Utility-Centric Retrieval in the LLM Era},
  author={Zhang, Hengran and Tang, Minghao and Bi, Keping and Guo, Jiafeng},
  journal={arXiv preprint arXiv:2604.08920},
  year={2026}
}