avatar

Clément Romac

Research Scientist / Ph.D. Student
Hugging Face / Inria Bordeaux (France)
clement[dot]romac[at]gmail[dot]com


About Me

Hi, I am Clément Romac. I am Research Scientist at Hugging Face and a last-year PhD student jointly supervised by Pierre-Yves Oudeyer (FLOWERS team, Inria Bordeaux) and Thomas Wolf (Hugging Face) studying how curiosity-driven RL can help grounding Large Language Models. My PhD is expected to finish around the end of 2025 and I am now looking for postdoctoral positions related to RL, curiosity-driven learning, or language grounding through embodied interactions. If you are interested, do not hesitate to reach out!

Previously, I worked as a Machine Learning Engineer at Weenove and as a Research Engineer in the FLOWERS team working on Automatic Curriculum Learning for Deep RL and Automated scientific discovery in complex systems (e.g. self-organizing). Apart from research, I have also been involved in various dissemination initiatives, in particular towards education (I am co-leading the development of the “ChatGPT explained in 5 minutes” series of videos that is used in French classrooms and now embedded in official trainings for workers from all French ministries). I am also the co-organizer of the Machine Learning Meetup group of Bordeaux.

Research Interests

My research focuses on Reinforcement Learning (RL) and how it can shape artificial agents’ knowledge. I notably study how Large Language Models (LLMs) can be grounded through online interactions with an environment using RL. I am interested in open-ended learning setups and my research often involves methods such as curiosity-driven RL or Automatic Curriculum Learning.

News

Publications

  1. RLDM
    WorldLLM: Improving LLMs' world modeling using curiosity-driven theory-making
    Guillaume Levy, Cedric Colas, Pierre-Yves Oudeyer, Thomas Carta, Clement Romac
    Multi-disciplinary Conference on Reinforcement Learning and Decision Making (RLDM) 2025
    PDF Code Poster

  2. ICML
    MAGELLAN: Metacognitive predictions of learning progress guide autotelic LLM agents in large goal spaces
    Loris Gaven, Thomas Carta, Clément Romac, Sylvain Lamprier, Olivier Sigaud, Pierre-Yves Oudeyer
    International Conference on Machine Learning (ICML), 2025.
    PDF Code Poster

  3. NAACL Findings
    Reinforcement Learning for Aligning Large Language Models Agents with Interactive Environments: Quantifying and Mitigating Prompt Overfitting
    Mohamed Salim Aissi, Clement Romac, Thomas Carta, Sylvain Lamprier, Pierre-Yves Oudeyer, Olivier Sigaud, Laure Soulier, Nicolas Thome
    NAACL Findings 2025.
    PDF Poster

  4. IMOL Workshop
    SAC-GLAM: Improving Online RL for LLM agents with Soft Actor-Critic and Hindsight Relabeling
    Loris Gaven, Clément Romac, Thomas Carta, Sylvain Lamprier, Olivier Sigaud, Pierre-Yves Oudeyer
    IMOL Workshop, NeurIPS 2024.
    PDF Poster

  5. ARLET Workshop
    Jack of All Trades, Master of Some, a Multi-Purpose Transformer Agent
    Quentin Gallouédec, Edward Beeching, Clément Romac, Emmanuel Dellandrea
    ARLET Workshop, ICML 2024.
    PDF Code Project Page Poster

  6. ICML
    Grounding Large Language Models in Interactive Environments with Online Reinforcement Learning
    Thomas Carta, Clément Romac, Thomas Wolf, Sylvain Lamprier, Olivier Sigaud, Pierre-Yves Oudeyer
    International Conference on Machine Learning (ICML), 2023.
    PDF Code Project Page Poster

  7. ICML
    TeachMyAgent: a Benchmark for Automatic Curriculum Learning in Deep RL
    Clément Romac, Rémy Portelas, Katja Hofmann, Pierre-Yves Oudeyer
    International Conference on Machine Learning (ICML), 2021.
    PDF Code Project Page Poster

  8. Meta Automatic Curriculum Learning
    Rémy Portelas, Clément Romac, Katja Hofmann, Pierre-Yves Oudeyer
    Preprint, 2020
    PDF Code Project Page

  9. Deep Recurrent Q-Learning vs Deep Q-Learning on a simple Partially Observable Markov Decision Process with Minecraft
    Clément Romac, Vincent Beraud
    Preprint, 2019
    PDF Code

Projects

  1. ChatGPT explained in 5 minutes
    A series of short videos explaining Large Language Models.
    Project Page

  2. Lamorel
    Lamorel is a Python library designed for RL practitioners eager to use Large Language Models (LLMs).
    Code

  3. Automated Discovery Tool
    A software for assisted and automated discovery of patterns in the exploration of complex systems.
    Code

  4. Interactive Deep RL Demo
    In-browser interactive demo of Deep Reinforcement Learning agents' adaptation to unknown 2D tasks.
    Project Page Code

Other stuff


Powered by Jekyll and Minimal Light theme.