•  

    Shape the Technology that will Shape your Future

    Nagymaros AI Retreat

    26 Oct - 1 Nov 2025

  • This year's theme:

    Deep Learning, Transformers, Diffusion

     

    Deep Learning, Simplicity Biases, Transformers

    This year we will learn some basic deep learning techniques while also talking about some of the fundamental concepts that make them so efficient: simplicity biases, overparametrisation, inductive biases. We will also look into the Transformers, the workhorse of modern language models and more.

    Diffusion language modelling

    A more advance topic we dive into is discrete diffusion-based language modeling. While diffusion has mainly been used for generating images and other continuous data, more recently efforts have been made to apply them to discrete generative AI tasks, such as language models. While the maths behind these models can be pretty complicated to understand even for those with university-level maths knowledge, there are some simpler formulations that may be easier to understand.

  • Sponsorship

    This year's event is sponsored by Reasonable. Thank you!

  • Agenda

    Saturday, 26 October

    • Around noon: arrival at Mihály utca
    • 13:00 Lunch
    • 15:00 Hike in Nagymaros

    27 - 31 October - project work

    • 9:00 breakfast
    • generally working a lot
    • also boardgames, a quiz, and more

    Saturday, 1 November - leaving home

    • retro meeting (mad, sad, glad format)
    • saying goodbye
  • Venues

    Nagymaros, Hungary

    Section image

    Mihály utca

    Our home for the week is a nice house in Nagymaros, close to the train station with lovely views over the river and surrounding hills. The house has communal areas, a kitchen, and rooms with 2-4 beds in each.

    Section image

    Piknik Manufaktúra

    To survive Wednesday's planned power cut, the retreat returns to Piknik Manufaktúra, the location of the first retreat in 2022

  • Speakers and Mentors

    full list of speakers TBD

    Section image

    Ferenc Huszár

    Professor of Machine Learning, Cambridge

    I studied Computer Engineering at BME (Budapest) and got into machine learning after joining a computational neuroscience research group. After PhD in Probabilistic Machine Learning at Cambridge, I worked in tech startups and companies for 10 years. I am now a Professor in Camridge University, where I built a research group on the theory of deep learning. I've been running AI retreats since 2022, and occasionally coached students for the new AI Olympiads.

    Section image

    Bea Benkő

     Doctoral Researcher in Machine Learning, ELTE, SZTAKI

    I have been studying Mathematics and Computer Science at ELTE (Budapest), and I am currently a final-year PhD student working on continual learning, representation learning, out-of-distribution detection, machine unlearning. After working for over two years as a software developer during my undergraduate studies, I joined the AI research group first at the Rényi Institute for Mathematics, and later at the HUN-REN Institute for Computer Science and Control (SZTAKI).

    Section image

    Attila Sulyok

    Doctoral Researcher in Machine Learning, PPTE

    I'm András Attila Sulyok, a Ph.D. student at PPKE ITK. I did my Master's and Bachelor's in Computer Engineering here too. My research focuses on reinforcement learning and generative modeling. I've done my fair share of coding and implementing experiments, and I'm pretty good at visualizing results. I've also become more skilled at debugging than I ever thought I'd need to be. I enjoy explaining concepts and working with geniuses (or is it genii?), though I'm still improving at both. This year's knot theory topic interests me – I sail as a hobby, so I've tied my share of knots.

    Section image

    Anna Mészáros

    PhD student in Machine Learning, Cambridge University

    I am lucky to have the best PhD supervisor.

  • Participants

    Full list of participants TBA

    Section image

    Hanna Jánoskúti

    I’m an 11th grader in the math class at Fazekas Mihály Gimnázium and member of the Hungarian national debate team. I’m interested in the mathematical theory and societal implications of AI, a topic I’ve explored through numerous debate competitions. Founder of Social Impact Award-winning EmpowherCareer and the Fazekas Debate Club.

    Section image

    Balázs Szilágyi

    A competitive programmer starting 12th grade at Veres Péter Gimnázium. Interested in optimisation problems, game theory and AI. Former Huawei Online ICPC Challenge prize winner, IOI, CEOI, and I(OAI|AIO) medalist.

    Section image

    Panni Toók

    Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

    Section image

    Vince Ungár

    I’m an 11th grader at Veres Péter Gimnázium, with my focus being on competitive programming. My main interests are AI, Mathematics and any problem-solving based task in general. IOAI and IAIO medalist.

    Section image

    Miklós Szászkay

    Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

    Section image

    Kseniia Strelbytska

    Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

    Section image

    Tamás Görömbey

    Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.

    Section image

    Zsolt Mihály

    Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat.