RecSys Challenge 2025

About top

The RecSys 2025 Challenge will be organized by Jacek Dąbrowski, Maria Janicka, Łukasz Sienkiewicz and Gergo Stomfai (Synerise), Dietmar Jannach (University of Klagenfurt, Austria), Marco Polignano ( University of Bari Aldo Moro, Italy), Claudio Pomo (Politecnico di Bari, Italy), Abhishek Srivastava (IIM Visakhapatnam, India), and Francesco Barile (Maastricht University, Netheralnds).

The challenge is designed to promote a unified approach to behavior modeling. Many modern enterprises rely on machine learning and predictive analytics for improved business decisions. Common predictive tasks in such organizations include recommendation, propensity prediction, churn prediction, user lifetime value prediction, and many others. A central piece of information that is used for these predictive tasks are logs of past behavior of users e.g., what they bought, what they added to their shopping cart, which pages they visited. Rather than treating these tasks as separate problems, we propose a unified modeling approach.

To achieve this, we introduce the concept of Universal Behavioral Profiles—user representations that encode essential aspects of each individual’s past interactions. These profiles are designed to be universally applicable across multiple predictive tasks, such as churn prediction and product recommendations. By developing representations that capture fundamental patterns in user behavior, we enable models to generalize effectively across different applications.


Challenge Task top

The objective of this challenge is to develop Universal Behavioral Profiles based on the provided data, which includes various types of events such as product buy, add to cart, remove from cart, page visit, and search query. These user representations will be evaluated based on their ability to generalize across a range of predictive tasks. The task of the challenge participants is to submit user representations, which will serve as inputs to a simple neural network architecture. Based on the submitted representations, models will be trained on several tasks, including some that are disclosed to participants, called "open tasks," as well as additional hidden tasks. The final performance score will aggregate results from all tasks. We iterate model training and evaluation automatically upon submission, the task of the participants is to submit universal user representations.

Open Tasks:

Hidden Tasks:

In addition to the open tasks, the challenge includes hidden tasks, which remain undisclosed during the competition. The purpose of these tasks is to ensure that submitted Universal Behavioral Profiles are capable of generalization rather than being fine-tuned for specific known objectives. Similar to the open tasks, the hidden tasks focus on predicting user behavior based on the submitted representations, but they introduce new contexts that participants are not explicitly optimizing for. After the competition concludes, the hidden tasks will be disclosed along with the corresponding code, allowing participants to replicate results.


Evaluation top

The primary metric by which we measure model performance is AUROC. Additionally, the performance of category propensity and product propensity models is evaluated based on the novelty and diversity of the results. In these cases, the task’s score is derived as a weighted sum of all metrics, specifically 0.8 × AUROC + 0.1 × Novelty + 0.1 × Diversity.

For each task, a leaderboard is created based on the respective task scores. The final score, which evaluates the overall quality of user representations and their ability to generalize, is determined by aggregating ranks from all per-task leaderboards using Borda count method. In this approach, each model's rank in a task leaderboard is converted into points, where a model ranked k-th among N participants receives (N - k) points. The final ranking is based on the total points accumulated across all tasks, ensuring that models performing well consistently across multiple tasks achieve a higher overall score.


DataSet top

The challenge organizers will publish an anonymized dataset containing real-world user interaction logs. All recorded interactions can be utilized to create Universal Behavioral Profiles; however, participants will be required to submit behavioral profiles only for a subset of users, which will be used for model training and evaluation.

The data will consist of four types of events and product attributes:

product_buy

add_to_cart

remove_from_cart

product_properties

page_visit

search_query

Event type Number of events
product_buy ~1 700 000
add_to_cart ~5 200 000
remove_from_cart ~1 700 000
page_visit ~150 000 000
search_query ~9 600 000
Total number of unique users ~19 000 000

Download

Universal Behavioral Modeling Dataset © 2025 by Synerise SA is licensed under Creative Commons Attribution-NonCommercial 4.0 International. To view a copy of this license, visit https://creativecommons.org/licenses/by-nc/4.0/
Download dataset

We provide a competition repository that includes a baseline solution and a training pipeline, allowing participants to run experiments on open tasks:
Repository

For further details, please refer to the dedicated website of Synerise.


Prize top



Participation and Data top

Registration & Data Access will open on [DATE]


Timeline top

When? What?
10 March, 2025 Start RecSys Challenge

Release dataset

10 April, 2025 Submission System Open

Leaderboard live

15 June, 2025 End RecSys Challenge
20 June, 2025 Final Leaderboard & Winners

EasyChair open for submissions

26 June, 2025 Code Upload

Upload code of the final predictions

7 July, 2025 Paper Submission Due
24 July, 2025 Paper Acceptance Notifications
2 August, 2025 Camera-Ready Papers
September 2025 RecSys Challenge Workshop

@ ACM RecSys 2025


Paper Submission Guidelines top

TBD


Organization top

Workshop Program Committee top

TBA