Tutorial Materials

Beyond Relevance: Utility-Centric Retrieval in the LLM Era — SIGIR 2026

← Back to Tutorial Home


Slides

Section 1

Introduction & Foundations

📄 Slides (PDF)

Section 2

What Is LLM-Centric Utility?

📄 Slides (PDF)

Section 3

Utility Modeling & Optimization

📄 Slides (PDF)

Section 4

Utility in Agentic RAG

📄 Slides (PDF)

Section 5

Open Problems & Q&A

📄 Slides (PDF)


Reading List

Tutorial Proposal: "Beyond Relevance: Utility-Centric Retrieval in the LLM Era"

Paper Repository: awesome-papers-on-Utility-focused-RAG (GitHub)

Section 1: Introduction and Foundations

1.1 RAG Foundations

1.2 Classical Relevance and Utility

1.3 User-Centric Utility in Web Search

1.5 Related Perspective: Denoising-First IR

Section 2: What Is LLM-Centric Utility?

2.2 Task-Specific Definitions of Utility

Factoid QA

Non-Factoid and Complex QA

Fact Verification and Classification

Code Generation

Tool and Skill Retrieval

2.3 LLM-Agnostic vs. LLM-Specific Utility

LLM-Agnostic Utility

LLM-Specific Utility

2.5 Pointwise and Set-Level Utility

Pointwise Utility

Set-Level Utility

2.6 Evidence Format and Presentation to LLMs

Section 3: Utility Modeling and Optimization Methods

3.1 Utility Signals

Signal A: Verbalized LLM Utility Judgment (no ground-truth answer required)

Signal B: Attention / Reader-Derived Signals (no ground-truth answer required)

Signal C: Likelihood and Perplexity (ground-truth answer required)

Signal D: Downstream Performance (ground-truth answer required)

3.2 Efficiency Ladder: From Small Candidate Sets to Corpus-Scale

Level 1: Small Candidate Sets (10–20) — High-Fidelity Judgments

Level 2: Medium Candidate Sets (dozens to hundreds) — Selectors and Rerankers

Level 3: Corpus-Scale — Utility-Oriented Retrievers

3.3 Evaluation: What Should We Measure?

Section 4: Utility in Agentic RAG

4.2 Query-side Utility

4.3 Document-side Utility

4.4 Action-side Utility


Citation

@inproceedings{zhang2026beyond,
  title={Beyond Relevance: Utility-Centric Retrieval in the LLM Era},
  author={Zhang, Hengran and Tang, Minghao and Bi, Keping and Guo, Jiafeng},
  booktitle={Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval},
  pages={5357--5361},
  year={2026}
}

← Back to Tutorial Home
TIME Group  |  Contact Keping Bi: bikeping@ict.ac.cn