Welcome from the L2IT director

• Jan Stark (L2I Toulouse, CNRS/IN2P3, UT3)

Room: Le Village, Auditorium

Welcome from the organizers; practicalities

• Sylvain Caillou (L2I Toulouse, CNRS/IN2P3, UT3)

Room: Le Village, Auditorium

Keynote Address: Foundation models for high energy physics

• Anna Hallin (Universität Hamburg)

Room: Le Village, Auditorium Foundation models are machine learning models designed to handle a wide range of datasets and tasks. After being pre-trained on a specific task on a specific dataset, these models can be fine-tuned for various downstream applications, including different tasks and datasets. Developing such models for physics data could significantly enhance performance in the field and substantially cut down the necessary training time and data requirements. In this talk, I will give an introduction to foundation models, provide an overview of the foundation models that exist for particle physics today, and discuss some challenges and outlooks for the future.

PolarBERT: a Foundation Model for Neutrino Telescope Data

• Inar Timiryasov (Niels Bohr Institute, University of Copenhagen)

Room: Le Village, Auditorium Neutrinos are elusive particles that require massive detectors for observation. The IceCube neutrino observatory at the South Pole is a cubic kilometer of Antarctic ice, instrumented with 5,160 digital optical modules. Its results play an essential role in both particle physics and astrophysics. Deep learning methods, such as graph neural networks, have been successfully applied to the steady stream of heterogeneous data IceCube is receiving. In this talk, we will present a foundation model for the IceCube data. It is trained in a self-supervised way without any data labeling. We further fine-tune this pretrained model for the downstream task of directional reconstruction of neutrino events. We show that this pretrained model significantly outperforms models trained from scratch. Remarkably, the foundation model does not require any knowledge of the IceCube detector geometry or characteristics of its electronics, since it extracts all the necessary information from the raw data.

Scientific Foundation Models for Computational Fluid Dynamics: threats and opportunities

• Luciano Drozda (CERFACS)
• Fernando Gonzalez (CERFACS)

Room: Le Village, Auditorium Scientific Foundation Models (SciFMs) hold the promise of accelerating numerical simulation of physical phenomena. In recent years, a myriad of SciFMs for weather forecasting have been proposed by major companies (e.g., Microsoft's ClimaX and Aurora) as well as research centers (e.g., ECMWF's AIFS). The development of SciFMs in other domains such as Computational Fluid Dynamics (CFD) has not yet reached similar maturity levels, though. In this presentation, we discuss threats and opportunities surrounding the training and deployment of SciFMs for CFD. On the one hand, weather forecasting historically benefits from open data sharing thanks to government-funded research. On the other hand, CFD community is mainly backed up by proprietary software from industry, which limits sharing of information and eventually impacts SciFM development. Still, recent initiatives in open CFD data sharing like BLASTNet give hope that open-sourced SciFMs for CFD will become available in a near future.

Small thinks big: transfer learning in KM3NeT/ORCA for neutrino event reconstruction

• Ivan Mozun

Room: Le Village, Auditorium This study explores using transformer models to analyze data from the KM3NeT/ORCA neutrino detector. Due to the current detector's size, reconstructing neutrino events is challenging. By training models on simulations for the full detector (115 detection units) and fine-tuning them on smaller configurations, significant performance improvements are achieved compared to models trained from scratch on very limited data. This approach also helps estimate the detector's sensitivity as it grows.

Automatic estimation of the wind turbine noise with recurrent neural networks

• ABDELAZYZ RKHISS (Doctorant à Grenoble INP)

Room: Le Village, Auditorium There is growing interest in the development of renewable energies, particularly wind power. However, wind turbines generate noise that can affect the sound environment of nearby residents. This study focuses on the isolation of wind turbine noise (WTN) level from the surrounding total noise. Our method is based on a Recurrent Neural Network (RNN) Architecture that captures temporal dependencies in the acoustic signal. This proposal is compared to Non-Negative Matrix Factorization (NMF) that has shown first promising results on a previous study on simulated sound scenes of wind turbine noise. Our approach relies on simple RNN Vanilla conducted using an end-to-end trained model, Gated Recurrent Network (GRU), and a Long Short TermMemory (LSTM) trained from scratch and compared in the same dataset to the NMF method.

Keynote Address: Gravitational waves coming at you from all directions

• Jonathan Gair (AEI Potsdam)

Room: Le Village, Auditorium In January of this year, the European Space Agency officially adopted the space-based gravitational wave detector, LISA, as a mission, to launch in 2035. LISA will open up a new band in the gravitational wave frequency spectrum, at millihertz frequencies. This band is expected to be very rich in sources, ranging from binaries of compact stars in our galaxy, to binaries involving supermassive black holes in the centres of galaxies, to stochastic backgrounds formed. In contrast to ground-based detectors, these sources will be overlapping in the LISA data in both time and frequency, posing a complex data analysis problem necessitating a simultaneous global-fit to all sources of all types. Data analysis will be further complicated by instrumental artefacts, including gaps and glitches in the data and an unknown and time-dependent instrumental noise level, and imperfect knowledge of signal models. In this talk, I will present the LISA data analysis challenge, describe the approaches that are being developed to tackle it using standard techniques, and highlight areas where novel machine learning approaches are or could be developed to improve the efficiency of the analysis.

Statistically principled learning for gravitational-wave inverse problems

• Alvin Chua (National University of Singapore)

Room: Le Village, Auditorium An important aspect of gravitational-wave astronomy is solving inverse problems, i.e., determining the properties of astrophysical sources from their gravitational signals. This involves the construction of complex forward models for possible signals by solving the equations of general relativity, as well as the use of these forward models in data-analysis algorithms to extract and characterise actual signals in detector data. Machine learning is increasingly used to confront modern challenges in these tasks, although it faces unique hurdles such as noise-dominated data and the need for high precision in modelling. It is also crucial to clarify how any proposed learning method relates to the existing Bayesian framework for solving gravitational-wave inverse problems. In this talk, I will discuss broad strategies for developing machine-learning methods that are tailored to the needs of the field while remaining defensible on scientific rigour and principle.

Neural density estimation for Galactic Binaries in LISA data analysis

• Natalia Korsakova (APC)

Room: Le Village, Auditorium The future space based gravitational wave detector LISA will observe millions of galactic binaries (GBs) constantly present in the data. A small fraction of this population will be individually resolved. One of the challenging tasks will be to estimate the parameters of resolvable GBs while disentangling them from each other and from other gravitational wave sources present in the data. This problem is referred to as a global fit. A Bayesian framework is often used to infer the parameters of the sources and their number. The efficiency of the sampling techniques strongly depends on the proposals, especially in the multi-dimensional parameter space. We show how to use neural density estimators, and in particular Normalising flows, in order to build proposals which significantly improve the convergence of sampling. We also demonstrate how these methods could help in building priors based on physical models and provide an alternative way to represent the catalogue of identified sources.

Beyond Gauss? A more accurate model for LISA astrophysical noise sources

• Riccardo Buscicchio (Universitá di Milano-Bicocca, Milan, IT)

Room: Le Village, Auditorium In this talk, we explore two assumptions ubiquitous in LISA data analysis: Gaussianity and stationarity of astrophysical noise sources (i.e. arising from source confusion). I will provide an overview on characterizations and parameter estimations techniques of both properties. I will review the most recent findings on the Galactic population of double white dwarfs and the extragalactic one of extreme mass-ratio inspirals. Employing a suite of advanced statistical techniques, we aim to enhance our understanding of the noise properties and improve the overall data modeling process.

Keynote Address: Deep learning and the global workspace theory

• Rufin VanRullen (Centre de Recherche Cerveau et Cognition (CerCo), Artificial and Natural Intelligence Toulouse Institute (ANITI))

Room: Le Village, Auditorium .

Semi-supervised multimodal representation learning through a global workspace

• Léopold Maytié (Université Toulouse III - Paul Sabatier, Toulouse, France & Artificial and Natural Intelligence Toulouse Institute (ANITI))

Room: Le Village, Auditorium Recent deep learning models can efficiently combine inputs from different modalities (e.g., images and text) and learn to align their latent representations, or to translate signals from one domain to another (as in image captioning, or text-to-image generation). However, current approaches mainly rely on brute-force supervised training over large multimodal datasets. In contrast, humans (and other animals) can learn useful multimodal representations from only sparse experience with matched cross-modal data. Here we evaluate the capabilities of a neural network architecture inspired by the cognitive notion of a ``Global Workspace'': a shared representation for two (or more) input modalities. Each modality is processed by a specialized system (pretrained on unimodal data, and subsequently frozen). The corresponding latent representations are then encoded to and decoded from a single shared workspace. Importantly, this architecture is amenable to self-supervised training via cycle-consistency: encoding-decoding sequences should approximate the identity function. For various pairings of vision-language modalities and across two datasets of varying complexity, we show that such an architecture can be trained to align and translate between two modalities with very little need for matched data (from 4 to 7 times less than a fully supervised approach). The global workspace representation can be used advantageously for downstream classification and cross-modal retrieval tasks and for robust transfer learning. Ablation studies reveal that both the shared workspace and the self-supervised cycle-consistency training are critical to the system's performance.

Learning how to design biomolecules using a neuro-symbolic architecture

• Thomas Schiex (Université fédérale de Toulouse, ANITI, INRAE)

Room: Le Village, Auditorium Designing requires to mix physical knowledge, experience accumulated from past designs and constraints defining design objectives. Proteins are large biomolecules that play crucial roles in all living organisms. They are linear polymers which can be described as a sequence in a 20 letter alphabet (one for each amino acid). They can therefore be represented as discrete objects. In water, most proteins fold in a 3D structure, defining continuous atomic coordinates. To design new proteins, we introduced an hybrid architecture that combines all above elements in a joint pairwise decomposable function over amino acid identities. Physics is represented as a force field, experience is extracted by Deep Learning from Nature's designs and design objectives represented as constraints. The resulting model is then passed to an automated reasoning prover to identify the most suitable chemical composition. The same architecture can learn how to play Sudoku from examples, w/o knowing the rules.

A graph-structured distance for heterogeneous datasets with meta variables

• Paul SAVES (DTIS, ONERA and Fédération ENAC ISAE-SUPAERO ONERA, Université de Toulouse, France)

Room: Le Village, Auditorium This talk presents a novel distance function and modeling framework for mixed-variable domains, effectively handling heterogeneous data with continuous, integer, and categorical variables, including meta variables that shape problem structure. This approach is presented in a paper that enhanced generalization and optimization in large representation models in science without partitioning data. A follow-up paper will extend this work by unifying surrogate modeling in architecture optimization, introducing graph-structured domains and partially decreed variables, with applications in green aeronautics via Bayesian optimization.

Challenges of heterogeneous data for building Linguistic Theory

• Anisia Popescu (LISN)

Room: Le Village, Auditorium Linguistics thrives on data, whether it stems from small highly controlled laboratory studies or from large heterogeneous datasets. Speech technology is increasingly providing new and varied tools to test linguistic theories (from sound change to second language learning) on large scale data. This, however, does not come without its challenges. In this presentation, we address one of the key challenges for testing linguistic theory (such as diverging voicing systems within language families), posed by heterogeneous data: the frequent absence of linguistically formatted metadata.

Round Table

• Anna Hallin (Universität Hamburg)
• David Roussel (Airbus)
• Daniel Murnane (Niels Bohr Institute, University of Copenhagen)
• Sylvain Caillou (L2I Toulouse, CNRS/IN2P3, UT3)
• François Lanusse (AIM, CNRS/CEA Paris-Saclay / Flatiron Institute)
• Jordi INGLADA (CESBIO, Université de Toulouse, CNES/CNRS/INRAe/IRD/UT3)
• Jonathan Gair (Max Planck Institute for Gravitational Physics)

Room: Le Village, Auditorium The round table is structured into three 30-minute discussion segments focused on the following topics: • Foundation Models in Science • Heterogeneous Data and Multimodal Representation Learning • Inverse Problem - Likelihood-Free simulation based approach

Keynote Address: RELEO - Representation Learning for Earth Observation

• Jordi INGLADA (CESBIO, Université de Toulouse, CNES/CNRS/INRAe/IRD/UT3)

Room: Le Village, Auditorium This talk will introduce the RELEO (REpresentation Learning for Earth Observation) project (2024-2028), a research chair of the Artificial and Natural Intelligence Toulouse Institute (ANITI). RELEO aims at developing new self-supervised representation learning methods to produce semantically meaningful probabilistic representations from high-dimensional multi-modal EO data. The originality of the approach lies on the use of prior knowledge from physical models into Deep Learning and thus proposing advances in uncertainty estimation and interpretability. Recent results will be presented: physics-constrained deep learning for biophysical parameter retrieval from satellite optical imagery and spectro-spatio-temporal encoders for large representation models for irregular and unaligned satellite image time series.

Identifying a piecewise affine signal from its nonlinear observation - application to DNA replication analysis

• Clara Lage (ENS de Lyon)

Room: Le Village, Auditorium An important challenge in DNA replication analysis is to recover a so-called timing profile, that contains important information about the replication dynamics, from nonlinear observations. We show that this challenge can be expressed as a nonlinear sparse coding inverse problem where the unknown timing profile is assumed to be piecewise affine. We propose a novel formalism and computational approach to harness it. In the noiseless case, we establish sufficient identifiability conditions for the timing profile,and prove that it is the solution of a non-convex optimization problem. We propose the DNA-inverse optimization method that provably finds the global solution to the nonlinear inverse problem for noisy signals. Besides being more computationally effective than the state-of-the-art optimization, our approach automatically recovers all configurations of the replication dynamics. This is crucial for DNA replication analysis, and was not possible with previous methods

Keynote Address: Multimodal Pretraining for Astrophysical Foundation Models

• François Lanusse (CNRS, UMR AIM / Flatiron Institute)

Room: Le Village, Auditorium Deep Learning has seen a recent shift in paradigm, from training specialized models on dedicated datasets, so-called Foundation Models, trained in a self-supervised manner on vast amounts of data and then adapted to solve specific tasks with state-of-the-art performance. This new paradigm has been exceptionally successful not only for large language models (LLMs) but in other domains such as vision models. However applications of this new approach in astrophysics are still very scarce, for reasons ranging from new architectures to the (surprising) lack of availability of suitable large scale datasets. In this talk, I will discuss our recent work on deploying such a Foundation Model approach in the context of representation learning for astronomical photometric and spectroscopic observations of galaxies. Our aim is to embed these inhomogeneous observations (e.g. different types of measurements, different instruments, etc...) into a shared embedding space, in a completely self-supervised manner. These embeddings can then be used for a variety of downstream applications (e.g. redshift estimation, morphology classification, estimating physical properties) with very simple machine learning methods and reach near optimal performance. More specifically, I will present our AstroCLIP method which allows us to align embeddings between data modalities, but also our more recent and ongoing work on building early-fusion multimodal models relying on modality-specific tokenizers and a joint large transformer model.

Galaxy detection with deep learning in radio data

• David Cornu (Observatoire de Paris | PSL)

Room: Le Village, Auditorium Astronomical facilities generate ever-increasing data volumes, rapidly approaching the exascale. In this talk, I will introduce YOLO-CIANNA, a deep-learning object detector for astronomical images, and present results over simulated 2D continuum images and HI emission cubes from the SKAO SDCs. I will then discuss how the method could be applied to data from the SKA precursor and how we could combine heterogeneous data from other types of surveys to build an instrumental-context-aware detector.

Searching for Dark Matter at the LHC with GNN

• Rafal MASELEK (LPSC (Grenoble))

Room: Le Village, Auditorium About 1/4 of the energy density of the visible Universe is comprised of Dark Matter (DM), an unfamiliar and elusive form of matter that is yet to be understood. DM particles can be detected by experiments at the Large Hadron Collider (LHC), however, such searches are very challenging. We propose a novel approach based on Graph Neural Networks, combining low- and high-level information to enhance searches for DM at the LHC. We evaluate the algorithm and provide limits on the benchmark DM model.

Salt: Multimodal, Multitask Models for the ATLAS Experiment

• Jackson Barr (UCL)

Room: Le Village, Auditorium In High Energy Physics, experimental data can range from low level hardware information from different sub-detectors to high level reconstructed physics events. To address the need for flexible, multimodal machine learning models within the ATLAS experiment, the Salt framework based upon PyTorch and Lightning has been developed. Salt was initially developed for the identification of heavy-flavour jets but has expanded to a wider range of tasks including searches for long lived particles, jet mass regression or vertex fitting.

Explaining Jet Flavour Taggers with Integrated Gradients

• Scott DeGraw (University College London)

Room: Le Village, Auditorium At the Large Hadron Collider (LHC), proton-proton collisions produce collimated streams of particles called jets created from particle decay chains. Identifying the particle that originated the jet (flavour tagging) is crucial. Modern taggers use deep learning models with features of the decay products as inputs. We show that integrated gradients reveal how these complex and opaque models use the characteristic features of the decay products to classify the particle that originated the jet.

Graph Neural Networks for track reconstruction in the ATLAS ITk detector

• Minh-Tuan Pham (University of Wisconsin-Madison)

Room: Le Village, Auditorium High-energy physics (HEP) experiments, e.g. ATLAS and CMS, collide opposing packs of particles and characterize the collision's final state. The innermost part of a detector consists of many sensors which detect the passage of a charged-particle by measuring its energy deposit. A tracking algorithm recreates from these measurements the trajectories of all particles, a computationally intensive process which the HEP community seeks to replace with machine learning alternatives. We present in this contribution a pipeline centered on Graph Neural Network (GNN) developed for the ATLAS Inner Tracker (ITk) achieving comparable performance to the traditional analogue. We describe ML techniques employed in this pipeline, and our solutions to the unique memory and time constraints associated with processing collision data at low level. Finally, we discuss the challenge of learning heterogeneous data collected by different sensor technologies in the ITk, and our ongoing effort to address it.

Large-scale deep-learning for weather and climate prediction

• Laure Raynaud (Météo-France)

Room: Le Village, Auditorium A new paradigm for weather and climate prediction has emerged recently : data-driven prediction models have achieved similar performances as standard physics-based models, thanks to an accurate (task-specific or task-agnostic) encoding of the data distribution. While these models are able to efficiently use relatively homogenous data, the next challenge to expand the capabilities of data-driven modeling is to fully exploit the vast range of atmospheric observations, characterized by spatio-temporal variations and heterogeneous outputs (point or spatial time series, vertical profiles, vertically integrated data, … ). An overview of existing LRM for weather & climate prediction will be presented, as well as early results for integrating heterogeneous data sources.

Keynote Address: Medium Range Weather Forecasting with Machine Learning

• Andrew, on behalf of the GraphCast team and GenCast team from Google DeepMind El-Kadi (Google DeepMind)

Room: Le Village, Auditorium The recent emergence of quality data, large scale compute and deep learning advancements has enabled an acceleration in the field of Machine Learning for Weather Forecasting. Today's talk centers on two pieces of work: GraphCast and GenCast, both Medium Range Global Weather Forecasting models. The former produces deterministic forecasts up to 10 days into the future, while the latter makes probabilistic forecasts, up to 15 days into the future. We cover: their state of the art results compared to the most accurate operational equivalents, explore subtle aspects of their training data heterogeneity and discuss the role they play as we move towards larger weather models.

Enhancing Ultrasound Localization Microscopy (ULM) with Spatio-Temporal Deep Learning

• Vassili PUSTOVALOV (Institut de recherche en informatique de Toulouse, Université Toulouse III - Paul Sabatier)

Room: Le Village, Auditorium The integration of Ultrasound Localization Microscopy (ULM) into ultrasound imaging has significantly improved resolution, providing precise insights into blood flow direction and velocity. However, despite its potential, ULM remains a complex and time-consuming technique, even as deep learning (DL) continues to drive its optimization. Current DL methods for microbubble (MB) superlocalization face challenges due to the use of high-resolution images in their networks, resulting in longer processing times compared to traditional ULM methods. Additionally, these methods often require arbitrary filtering of results before integration into tracking algorithms. To address these challenges, our study introduces a novel DL approach inspired by single-molecule localization techniques. Our 3D convolutional neural network, called 3DML-ResNet, enables fast and scalable superlocalization while providing explicit estimation of the number of MBs present in each image.

Preprocessing arbitrarily structured data for AI with Awkward Array

• Vangelis Kourlitis (Technical University of Munich)

Room: Le Village, Auditorium Processing heterogeneous multimodal data presents challenges. These datasets feature complex, irregular structures due to nested or variable-sized outputs from different sensors, or due to missing data values. The data are typically of mixed types, complicating the preprocessing steps required before they can be fed into algorithms like multimodal representation models. AI practitioners must manage these complexities effectively. Awkward Array is a Python library designed to process arbitrarily structured data. Operating on an array-programming paradigm, it allows users to manipulate data using NumPy-like syntax. Awkward Array also includes GPU-accelerated kernels, enabling the preprocessing of complex data directly on modern hardware accelerators, which can significantly optimize the training process and reduce data transfer latency to the device. We introduce the Awkward Array library and provide examples that demonstrate its usage, highlighting its potential as an AI preprocessor.

Leveraging AI in computational physics with NVIDIA Modulus and TorchFort

• Frédéric Parienté (NVIDIA)
• Corentin Lapeyre (NVIDIA)

Room: Le Village, Auditorium NVIDIA supports the scientific community in leveraging data-driven and AI approaches in computational physics workflows. This talk will showcase how researchers use NVIDIA's open-source libraries like Modulus to integrate learning methods with scientific solvers. Specifically, it will focus on recent results that facilitate AI in-the-loop approaches and enable on-the-fly training and inference during computation thanks to the open-source library TorchFort. Finally, it will introduce a blueprint that new users can follow to replicate this in their simulation workflows, unlocking new areas of research without compromising the speed of their high-performance solvers.

Optimizing PyTorch: Accelerating Training and Inference with Compilation, Custom Kernels, and Beyond

• Alvaro Moran (Hugging Face)

Room: Le Village, Auditorium In this talk, we'll explore cutting-edge techniques to optimize both training and inference in PyTorch, enabling faster, more efficient model execution. We'll dive into the power of PyTorch's `torch.compile` to accelerate workflows by fusing operations and generating optimized code, reducing runtime overhead. Additionally, we'll cover the use of custom kernels with tools like Triton, Pallas and CUDA, allowing fine-grained control over GPU and TPU execution for performance-critical tasks. Beyond that, we'll have an overview on various methods like mixed precision, memory optimization strategies, and distributed training, all aimed at achieving optimal performance for large-scale machine learning models.

Keynote Address: A causal perspective on reliable and interpretable representation learning

• Michel Besserve (MPI for Intelligent Systems, Tuebingen, Germany)

Room: Le Village, Auditorium Artificial intelligence is increasingly relied on to assist with complex tasks by leveraging vast amounts of data. Building useful representations is a core ingredient to the performance of such systems, and arguably goes beyond the mere extraction of statistical information in observed data. One way to express desiderata for such representations using modifications to the data generation process: we "understand" and trust it when we comprehend its behavior in response to plausible and meaningful changes in the environment it is exposed to. Causality offers a comprehensive framework for modeling these changes through the concepts of interventions and counterfactuals. Focusing mainly on generative models, I will illustrate how causal desiderata can be used to guide representation learning, such that important aspects of the ground truth data generation process can be recovered. I will further elaborate on how these principles can also be applied in contexts that leverage domain knowledge in the form of scientific simulations instead of real data, and highlight some open questions raised by the use of AI in Science.

Closing words & Farewell

• Jan Stark (L2I Toulouse, CNRS/IN2P3, UT3)

Room: Le Village, Auditorium