The RecSys 2026 Challenge will be organized by Seungheon Doh (Korea Advanced Institute of Science and Technology, South Korea), Sergio Oramas (Pandora/SiriusXM), Bruno Sguerra (Deezer Research), Abhinav Bohra (Amazon), Claudio Pomo (Politecnico di Bari, Italy), and Francesco Barile (Maastricht University, Netherlands).

The RecSys Challenge 2026: Music-CRS focuses on the evolving landscape of music discovery, where static recommendation lists are being replaced by dynamic, conversational interactions. As users increasingly interact with AI through natural language, there is a critical need for systems that can seamlessly integrate Natural Language Understanding (NLU) with high-precision Recommender Systems (RecSys). This challenge aims to push the boundaries of how AI understands nuanced user preferences, explores musical tastes through dialogue, and provides contextually relevant track recommendations.

By utilizing the TalkPlayData-Challenge dataset, a large-scale conversation resource generated through an advanced agentic pipeline, we invite the global research community to tackle the complexities of multi-turn preference elicitation. As a research community-driven initiative, the challenge dataset features LLM-generated multi-turn dialogues paired with music metadata and user-item interaction data derived from publicly available research datasets. No SiriusXM or Deezer user data is included in the dataset. This challenge serves as a bridge between the NLP and RecSys communities, fostering next-generation innovation in interactive and personalized music information retrieval.

Challenge Task ^top

The primary goal is to develop a Conversational Music Recommendation system that acts as an intelligent agent capable of navigating user tastes through dialogue.

Main Task: Conversational Music Recommendation. The system must understand user music preferences from previous conversation turns and user profiles to recommend relevant tracks from a catalog while generating natural, helpful responses.

• Recommendation (Ranked Retrieval). Given the dialogue history and user context, the system must output a ranked list of the top 20 music tracks from the provided catalog. This requires capturing both explicit requests and latent preferences (e.g., mood, genre-specific nuances) within the conversation.
• Response Generation. The system must generate a natural language response to accompany the recommendations. This response should justify the picks based on the user's profile or previous turns and maintain a coherent conversational flow.

Evaluation ^top

The challenge employs a multi-dimensional evaluation framework to assess both what a system recommends and how it communicates those recommendations. The final score combines multiple dimensions, and each individual dimension contributes to the overall ranking. Recommendation quality serves as the primary anchor of the evaluation, while diversity and response-quality dimensions capture complementary aspects of system performance.

Development vs. blind evaluation. The public evaluator supports transparent development-set evaluation. Blind A serves as an interim leaderboard for the challenge, while Blind B is the final evaluation split. Blind Dataset B will be released one week before the end of the challenge, and the final leaderboard will be computed on Blind Dataset B.

Dimension	What it measures	How it is computed	Role in evaluation
nDCG@20	Ranking quality of the recommended tracks.	Computed from the ranked list of predicted tracks against the ground-truth relevant item. Higher-ranked correct recommendations receive more credit.	Primary recommendation metric.
Catalog Diversity	How broadly a system covers the music catalog.	Number of unique recommended tracks across all predictions divided by the total catalog size.	Complementary diversity indicator.
Lexical Diversity	How varied the generated language is.	Measured with Distinct-2, i.e., unique bigrams divided by total bigrams across generated responses.	Complementary response-generation indicator.
LLM-as-a-Judge	Quality of the generated explanation.	Blind-set responses are judged by a Gemini model used as an automatic judge. The judge evaluates two text-only dimensions: Personalization and Explanation Quality. These dimensions evaluate the written response independently from recommendation accuracy. To preserve the integrity of the blind evaluation, we disclose the judge family but do not publish the evaluation prompt.	Blind-set response-quality evaluation.

Aggregation policy. For the public evaluator, retrieval metrics are computed for each session-turn prediction, averaged within each turn, and then macro-averaged across turns. Diversity metrics are computed globally over the full prediction file.

To ensure transparency and fairness, all dimensions used in the evaluation are disclosed publicly. While we do not publish the exact aggregation formula on the website, participants should note that every listed dimension contributes to the final score. For the response-quality evaluation, we disclose that the automatic judge is based on Gemini, but we do not publish the evaluation prompt.

The public evaluator supports transparent development-set evaluation, while blind-set scoring is performed on the official leaderboard infrastructure.

Dataset: TalkPlayData-Challenge ^top

The challenge utilizes TalkPlayData-Challenge, a synthetic conversational music recommendation dataset featuring multi-turn dialogues and music listening history. The dataset is split into four subsets: Train, Development, Blind A, and Blind B.

The leaderboard evaluation proceeds in two blind stages. Blind A supports the interim leaderboard during the main phase of the challenge. Blind B is the final hidden evaluation set, released one week before the end of the challenge, and the final leaderboard is based on Blind B.

Dataset Components

• Conversation Dataset: Includes multi-turn dialogues (avg. 8 turns) with fields for session_id, user_id, turn_number, and the utterance text.
• Track Metadata: Contains information for over 1 million tracks, including track_id, track name, artist name, album name, tags (genre/mood), and release date.
• User Profiles: Provides demographic and historical context, including user_id, age, gender, country, and a list of track_ids representing listening history.
• Pre-extracted Embeddings: To ensure focus on model architecture, we provide pre-computed multimodal track embeddings (audio/content) and user embeddings (collaborative filtering).

For further details, please refer to the dedicated website .

The following resources are provided to support participation in the challenge.

Conversation Datasets ^top

• Train / Development Dataset: TalkPlayData-Challenge-Dataset
• Blind Dataset A: TalkPlayData-Challenge-Blind-A
• Blind Dataset B: Link will be published at release time.

Shared Recommendation Resources ^top

The following resources are shared across Train, Development, Blind A, and Blind B.

• User Metadata: TalkPlayData-Challenge-User-Metadata
• Track Metadata: TalkPlayData-Challenge-Track-Metadata
• User Embeddings: TalkPlayData-Challenge-User-Embeddings
• Track Embeddings: TalkPlayData-Challenge-Track-Embeddings

Code and Participation ^top

• Registration Website: Music-CRS Challenge Website
• Baseline Code: music-crs-baselines
• Evaluation Code: music-crs-evaluator

Prize ^top

• First team from the participants - $1000
• First academic team - $1000

• Seungheon Doh — KAIST, South Korea
• Sergio Oramas — Pandora/SiriusXM
• Bruno Sguerra — Deezer Research
• Abhinav Bohra — Amazon
• Claudio Pomo — Politecnico di Bari, Italy
• Francesco Barile — Maastricht University, Netherlands