The RecSys 2026 Challenge will be organized by Seungheon Doh (Korea Advanced Institute of Science and Technology, South Korea), Sergio Oramas (Pandora/SiriusXM), Bruno Sguerra (Deezer Research), Abhinav Bohra (Amazon), Claudio Pomo (Politecnico di Bari, Italy), and Francesco Barile (Maastricht University, Netherlands).

The RecSys Challenge 2026: Music-CRS focuses on the evolving landscape of music discovery, where static recommendation lists are being replaced by dynamic, conversational interactions. As users increasingly interact with AI through natural language, there is a critical need for systems that can seamlessly integrate Natural Language Understanding (NLU) with high-precision Recommender Systems (RecSys). This challenge aims to push the boundaries of how AI understands nuanced user preferences, explores musical tastes through dialogue, and provides contextually relevant track recommendations.

By utilizing the TalkPlayData-Challenge dataset, a large-scale conversation resource generated through an advanced agentic pipeline, we invite the global research community to tackle the complexities of multi-turn preference elicitation. As a research community-driven initiative, the challenge dataset features LLM-generated multi-turn dialogues paired with music metadata and user-item interaction data derived from publicly available research datasets. The dataset does not include proprietary or confidential data provided directly by SiriusXM, Deezer, or any challenge sponsor. This challenge serves as a bridge between the NLP and RecSys communities, fostering next-generation innovation in interactive and personalized music information retrieval.

Challenge Task ^top

The primary goal is to develop a Conversational Music Recommendation system that acts as an intelligent agent capable of navigating user tastes through dialogue.

Main Task: Conversational Music Recommendation. The system must understand user music preferences from previous conversation turns and user profiles to recommend relevant tracks from a catalog while generating natural, helpful responses.

• Recommendation (Ranked Retrieval). Given the dialogue history and user context, the system must output a ranked list of the top 20 music tracks from the provided catalog. This requires capturing both explicit requests and latent preferences (e.g., mood, genre-specific nuances) within the conversation.
• Response Generation. The system must generate a natural language response to accompany the recommendations. This response should justify the picks based on the user's profile or previous turns and maintain a coherent conversational flow.

Candidate catalog rule. During inference, recommender systems must retrieve candidates from the entire track catalog. Participants must not filter, subset, or restrict tracks using track_split_types or any other mechanism. For BM25/BERT baselines, the configuration must include track_split_types: ["all_tracks"]; submissions that do not use all_tracks may be considered invalid.

Evaluation ^top

The challenge employs a multi-dimensional evaluation framework to assess both what a system recommends and how it communicates those recommendations. The official composite score is:

Score = 0.50 x nDCG@20 + 0.10 x Catalog Diversity + 0.10 x Lexical Diversity + 0.30 x LLM-as-a-Judge

Development vs. blind evaluation. The public evaluator supports transparent development-set evaluation. Blind A served as an interim leaderboard phase, while Blind B is the official final evaluation split. The final ranking is based on the Blind B leaderboard.

Dimension	Weight	What it measures	How it is computed	Role in evaluation
nDCG@20	0.50	Ranking quality of the recommended tracks.	Computed from the ranked list of predicted tracks against the ground-truth relevant item. Higher-ranked correct recommendations receive more credit.	Primary recommendation metric.
Catalog Diversity	0.10	How broadly a system covers the music catalog.	Number of unique recommended tracks across all predictions divided by the total catalog size.	Complementary diversity indicator.
Lexical Diversity	0.10	How varied the generated language is.	Measured with Distinct-2, i.e., unique bigrams divided by total bigrams across generated responses.	Complementary response-generation indicator.
LLM-as-a-Judge	0.30	Quality of the generated explanation.	Blind-set responses are judged by a Gemini model used as an automatic judge. The judge evaluates two text-only dimensions: Personalization and Explanation Quality. These dimensions evaluate the written response independently from recommendation accuracy. To preserve the integrity of the blind evaluation, we disclose the judge family but do not publish the evaluation prompt.	Blind-set response-quality evaluation.

Aggregation policy. Each conversation turn has exactly one ground-truth track. The evaluation reports nDCG@1, nDCG@10, and nDCG@20, with nDCG@20 as the primary retrieval metric. Results are macro-averaged across sessions and turns. LLM judge scores use a 1-5 integer scale and are normalized to [0, 1] before being weighted in the composite score.

To preserve the integrity of the blind evaluation, the judge family is disclosed, but the detailed evaluation prompt is not published. Blind-set scoring is performed on the official Codabench leaderboard infrastructure.

Dataset: TalkPlayData-Challenge ^top

The challenge uses TalkPlayData-Challenge, a large-scale multi-turn dialogue dataset for conversational music recommendation, with pre-extracted multimodal track and user embeddings provided. It is based on the publicly available TalkPlayData dataset and was prepared for this research challenge by members of the organizing committee affiliated with KAIST.

The leaderboard evaluation proceeds in two blind stages. Blind A supported the interim leaderboard during the main phase of the challenge. Blind B is the final hidden evaluation set and the final leaderboard is based on Blind B. The dataset is released under CC BY-NC 4.0 for non-commercial research use only; redistribution outside the scope of the challenge is not permitted.

Dataset Components

• Conversation Datasets: Include multi-turn dialogue instances for each session_id and turn_number pair in the corresponding split.
• Track Metadata: Provides the official catalog and valid track identifiers for recommendation outputs.
• User Metadata: Provides user-side context where available. Blind B includes cold-start cases where user IDs and profiles are not provided.
• Pre-extracted Embeddings: To ensure focus on model architecture, we provide pre-computed multimodal track embeddings (audio/content) and user embeddings (collaborative filtering).

For further details, please refer to the dedicated website .

The following resources are provided to support participation in the challenge.

Conversation Datasets ^top

• Train / Development Dataset: TalkPlayData-Challenge-Dataset
• Blind Dataset A: TalkPlayData-Challenge-Blind-A
• Blind Dataset B: TalkPlayData-Challenge-Blind-B

Shared Recommendation Resources ^top

The following resources are shared across Train, Development, Blind A, and Blind B.

• User Metadata: TalkPlayData-Challenge-User-Metadata
• Track Metadata: TalkPlayData-Challenge-Track-Metadata
• User Embeddings: TalkPlayData-Challenge-User-Embeddings
• Track Embeddings: TalkPlayData-Challenge-Track-Embeddings

Code and Participation ^top

• Official Codabench Competition: RecSys Challenge 2026: Conversational Music Recommendation
• Registration Website: Music-CRS Challenge Website
• Baseline Code: music-crs-baselines
• Evaluation Code: music-crs-evaluator

Prize ^top

The total prize pool is $2,000 USD, funded by external sponsors including SiriusXM and Deezer. Sponsors provide financial support only and do not control or influence the dataset, evaluation methodology, rankings, or outcomes of the challenge.

• Academic Track - 1st Place: $1,000
• Industry Track - 1st Place: $1,000

Winners are responsible for any applicable taxes in their jurisdiction.

Registration and Teams ^top

• One account is allowed per participant. Multiple accounts may result in disqualification.
• Each team must register a single account and submit from that account only.
• The maximum team size is 10 members.
• Team mergers must be reported to the organizers before the challenge ends.
• Phase B requires frozen team membership. Solo participants must register as a one-person team. Team registration is available through the official team registration form.

Participation Tracks ^top

• Academic Track: at least 50% of team members are currently enrolled students or academic staff.
• Industry Track: all other teams.
• Tracks are ranked separately based on the final Blind B leaderboard.

Submission Format and Limits ^top

• Submissions must be uploaded as submission.zip and must contain a file named exactly prediction.json. Any other filename will fail scoring.
• prediction.json must provide predictions for every (session_id, turn_number) pair in the evaluation set.
• Each prediction must include an ordered list of up to 20 unique valid track IDs and a predicted_response string.
• Blind A: 10 submissions per day, 500 submissions total.
• Blind B: 3 submissions total; this is the final phase.
• Participants must manually select which Blind B submission should be considered for the final leaderboard evaluation.
• Top-ranked teams may be required to upload code for verification of results.

These terms summarize the official Codabench Terms & Conditions for the Music-CRS Challenge 2026. By registering, participants agree to comply with the official rules.

• The challenge is open to researchers, students, and practitioners, except those directly involved in organizing the challenge.
• Participants are responsible for ensuring that their participation does not violate agreements with employers or institutions and must comply with applicable laws and regulations.
• Participants are expected to act in good faith. Cheating, plagiarism, prompt injection, output contamination, attempts to manipulate the evaluation process, duplicate accounts, or multi-team participation may result in deletion of submissions or disqualification.
• Code and data must not be shared outside the team unless they are openly shared with all participants through official channels.
• Registration information is collected only for challenge administration. Participants may use a pseudonym on public leaderboards, but winners must provide their real identity to receive prizes.
• The dataset and challenge are provided "as is" without warranties of any kind. Participation is voluntary and at the participant's own risk.
• The organizers may update the guidelines as necessary and may modify, postpone, or cancel the challenge if unforeseen circumstances arise.

Official reference: Codabench RecSys Challenge 2026 .

• Seungheon Doh — KAIST, South Korea
• Sergio Oramas — Pandora/SiriusXM
• Bruno Sguerra — Deezer Research
• Abhinav Bohra — Amazon
• Claudio Pomo — Politecnico di Bari, Italy
• Francesco Barile — Maastricht University, Netherlands