The RecSys 2025 Challenge will be organized by Jacek Dąbrowski, Maria Janicka, Łukasz Sienkiewicz and Gergo Stomfai (Synerise), Dietmar Jannach (University of Klagenfurt, Austria), Marco Polignano ( University of Bari Aldo Moro, Italy), Claudio Pomo (Politecnico di Bari, Italy), Abhishek Srivastava (IIM Visakhapatnam, India), and Francesco Barile (Maastricht University, Netheralnds).
The challenge is designed to promote a unified approach to behavior modeling. Many modern enterprises rely on machine learning and predictive analytics for improved business decisions. Common predictive tasks in such organizations include recommendation, propensity prediction, churn prediction, user lifetime value prediction, and many others. A central piece of information that is used for these predictive tasks are logs of past behavior of users e.g., what they bought, what they added to their shopping cart, which pages they visited. Rather than treating these tasks as separate problems, we propose a unified modeling approach.
To achieve this, we introduce the concept of Universal Behavioral Profiles—user representations that encode essential aspects of each individual’s past interactions. These profiles are designed to be universally applicable across multiple predictive tasks, such as churn prediction and product recommendations. By developing representations that capture fundamental patterns in user behavior, we enable models to generalize effectively across different applications.
The objective of this challenge is to develop Universal Behavioral Profiles based on the provided data, which includes various types of events such as product buy, add to cart, remove from cart, page visit, and search query. These user representations will be evaluated based on their ability to generalize across a range of predictive tasks. The task of the challenge participants is to submit user representations, which will serve as inputs to a simple neural network architecture. Based on the submitted representations, models will be trained on several tasks, including some that are disclosed to participants, called "open tasks," as well as additional hidden tasks. The final performance score will aggregate results from all tasks. We iterate model training and evaluation automatically upon submission, the task of the participants is to submit universal user representations.
Open Tasks:
Hidden Tasks:
In addition to the open tasks, the challenge includes hidden tasks, which remain undisclosed during the competition. The purpose of these tasks is to ensure that submitted Universal Behavioral Profiles are capable of generalization rather than being fine-tuned for specific known objectives. Similar to the open tasks, the hidden tasks focus on predicting user behavior based on the submitted representations, but they introduce new contexts that participants are not explicitly optimizing for. After the competition concludes, the hidden tasks will be disclosed along with the corresponding code, allowing participants to replicate results.
The primary metric by which we measure model performance is AUROC. Additionally, the performance of category propensity and product propensity models is evaluated based on the novelty and diversity of the results. In these cases, the task’s score is derived as a weighted sum of all metrics, specifically 0.8 × AUROC + 0.1 × Novelty + 0.1 × Diversity.
For each task, a leaderboard is created based on the respective task scores. The final score, which evaluates the overall quality of user representations and their ability to generalize, is determined by aggregating ranks from all per-task leaderboards using Borda count method. In this approach, each model's rank in a task leaderboard is converted into points, where a model ranked k-th among N participants receives (N - k) points. The final ranking is based on the total points accumulated across all tasks, ensuring that models performing well consistently across multiple tasks achieve a higher overall score.
The challenge organizers will publish an anonymized dataset containing real-world user interaction logs. All recorded interactions can be utilized to create Universal Behavioral Profiles; however, participants will be required to submit behavioral profiles only for a subset of users, which will be used for model training and evaluation.
The data will consist of four types of events and product attributes:
product_buy
add_to_cart
remove_from_cart
product_properties
page_visit
search_query
Event type | Number of events |
---|---|
product_buy | ~1 700 000 |
add_to_cart | ~5 200 000 |
remove_from_cart | ~1 700 000 |
page_visit | ~150 000 000 |
search_query | ~9 600 000 |
Total number of unique users | ~19 000 000 |
Universal Behavioral Modeling Dataset © 2025 by Synerise SA is licensed under Creative Commons
Attribution-NonCommercial 4.0 International. To view a copy of this license, visit
https://creativecommons.org/licenses/by-nc/4.0/
Download dataset
We provide a competition repository that includes a baseline solution and a training pipeline,
allowing participants to run experiments on open tasks:
Repository
For further details, please refer to the dedicated website of Synerise.
When? | What? |
---|---|
10 March, 2025 |
Start RecSys Challenge
Release dataset |
10 April, 2025 |
Submission System Open
Leaderboard live |
15 June, 2025 | End RecSys Challenge |
20 June, 2025 |
Final Leaderboard & Winners
EasyChair open for submissions |
26 June, 2025 |
Code Upload
Upload code of the final predictions |
7 July, 2025 | Paper Submission Due |
24 July, 2025 | Paper Acceptance Notifications |
2 August, 2025 | Camera-Ready Papers |
September 2025 | RecSys Challenge Workshop |