Paris workshop on Bayesian Deep Learning for Cosmology and Time Domain Astrophysics

20 juin 2022, 07:00 → 24 juin 2022, 17:20 Europe/Paris

Buffon Amphitheater (APC laboratory, Université Paris Cité)

Machine learning attracts a lot of interest in the fields of cosmology and time-domain astronomy and may potentially lead to major breakthroughs. Its adoption by the scientific community has been increasing dramatically in the past few years. Current progress in the machine learning community can simultaneously bring a lot to ours and must be monitored closely.

Among developments of interest for the astronomy community, probabilistic machine learning models, especially Bayesian neural networks, bring an estimation of uncertainty and combine deep neural networks architecture with Bayesian inference. 

Cosmologists also rely a lot on forward modelling and often face intractable likelihoods. Modern techniques like simulation-based inference and differentiable programming can be very valuable for model selection and model parameters inference.

Time series analyses have relied for some time on the use of recurrent neural networks, more recently on transformers. Some other analyses have been using convolutional neural networks, or graph neural networks. The workshop will explore if new architectures like graph transformers could be relevant in some fields, and how such networks can be made probabilistic.

This workshop will give the participants the opportunity to learn more about these emerging methods and how to use and exploit them in their research. The  program includes invited lectures and tutorials from major computer science experts and invited and contributed talk, and poster sessions aimed at sharing experience between physicists on the practical applications of machine learning. It is intended for researchers and students that are familiar with machine learning, use this type of algorithms for their own work, and want to learn about the advanced techniques related to Bayesian deep learning.

This workshop is part of the LSSTC Enabling Science effort and will give opportunity to younger scientists to apply for a grant covering lodging and part of the conference fees. Application can be made through a dedicated page on the menu on the left 👈.

Workshop program

• Mon 20 – "School day" with 4 x 90 min interactive sessions
• Tue 21–Fri 24 June – Workshop with keynote speakers, contributed talks, tutorials, round tables and lightning talks.

Workshop topics

• Cosmology and time-domain astro applications of Bayesian deep networks
• Methods for quantifying models uncertainty
• Anomaly and outlier detection
• Simulation-based and likelihood-free inference
• Probabilistic ML frameworks
• Use of Bayesian deep learning outside of academia
• Ethics of large-scale machine learning

Important dates

• April 19th 2022 – pre-registration, registration and call for contributions start
• May 18th 2022 – contributions & grant application deadline for pre-registered young scientists
• June 6th 2022 – end of registration

Confirmed speakers

• Anja Butter, ITP Heidelberg, Germany
• Colin Caroll, Google, USA
• Jean-Gabriel Ganascia, LIP6, Paris, France
• Stephen Green, MPI, Potsdam, Germany
• Alan Heavens, Imperial College, London, UK
• Tomasz Kacprzak, ETH Zurich / PSI, Switzerland
• Junpeng Lao, Google, Switzerland
• Ashley Villar, Penn State University, USA
• Ben Wandelt, IAP, Paris, France

lundi 20 juin

Break: Discussions around coffee & tea - Registration

Organisers: Welcome address - BDL School

Présidents de session: Alexandre Boucaud (APC / IN2P3), Eric Aubourg (APC)

School day welcome slides

Workshop welcome slides (tuesday)

Tutorials: BDL school

Président de session: Alan Heavens ( Imperial College London)

AlanHeavens

• Introduction-to-Bayes-Alan-Heavens.pdf
• Straight Line Gibbs.ipynb
• Straight Line HMC.ipynb
• Straight Line MH MCMC.ipynb

Junpeng Lao

• Bayesian_Time_Series_Analysis_with_TFP_on_JAX_(Part_1).ipynb
• Bayesian_Time_Series_Analysis_with_TFP_on_JAX_(Part_2).ipynb
• Bayesian Time Series Analysis with TFP on JAX.pdf

1 Introduction to Bayesian Modeling

Orateur: Alan Heavens ( Imperial College London)

Introduction-to-Bayes-Alan-Heavens.pdf

Straight Line Gibbs.ipynb

Straight Line HMC.ipynb

Straight Line MH MCMC.ipynb

Break: Coffee break

Tutorials: BDL School

Président de session: Junpeng Lao (Google Zurich)

AlanHeavens

• Introduction-to-Bayes-Alan-Heavens.pdf
• Straight Line Gibbs.ipynb
• Straight Line HMC.ipynb
• Straight Line MH MCMC.ipynb

Junpeng Lao

• Bayesian_Time_Series_Analysis_with_TFP_on_JAX_(Part_1).ipynb
• Bayesian_Time_Series_Analysis_with_TFP_on_JAX_(Part_2).ipynb
• Bayesian Time Series Analysis with TFP on JAX.pdf

2 Bayesian Time Series Analysis with TFP on JAX

Orateur: Junpeng Lao (Google Zurich)

Bayesian time series analysis – Colab notebook – part 1

Bayesian time series analysis – Colab notebook – part 2

Bayesian Time Series Analysis with TFP on JAX

Break: Lunch (on your own)

Tutorials: BDL school

Président de session: Francois Lanusse ({CNRS}UMR7158)

AlanHeavens

• Introduction-to-Bayes-Alan-Heavens.pdf
• Straight Line Gibbs.ipynb
• Straight Line HMC.ipynb
• Straight Line MH MCMC.ipynb

Junpeng Lao

• Bayesian_Time_Series_Analysis_with_TFP_on_JAX_(Part_1).ipynb
• Bayesian_Time_Series_Analysis_with_TFP_on_JAX_(Part_2).ipynb
• Bayesian Time Series Analysis with TFP on JAX.pdf

3 Probabilistic Deep Learning with TFP and JAX/Flax

Orateur: Francois Lanusse ({CNRS}UMR7158)

Notebook

Slides

Break: Coffee break

Tutorials: BDL school

Président de session: Benjamin Wandelt (IAP)

AlanHeavens

• Introduction-to-Bayes-Alan-Heavens.pdf
• Straight Line Gibbs.ipynb
• Straight Line HMC.ipynb
• Straight Line MH MCMC.ipynb

Junpeng Lao

• Bayesian_Time_Series_Analysis_with_TFP_on_JAX_(Part_1).ipynb
• Bayesian_Time_Series_Analysis_with_TFP_on_JAX_(Part_2).ipynb
• Bayesian Time Series Analysis with TFP on JAX.pdf

4 Likelihood-Free Inference and Application to Cosmology

Orateur: Benjamin Wandelt (IAP)

mardi 21 juin

08:30 Discussions around coffee & tea - Registration

Organisers: Welcome address – Workshop

Présidents de session: Alexandre Boucaud (APC / IN2P3), Cécile Roucelle (APC), Eric Aubourg (APC)

School day welcome slides

Workshop welcome slides (tuesday)

Uncertainties

5 When (not) to use uncertainties for ML in particle physics

Orateur: Anja Butter (LAL Orsay)

Bayes_Deep.key

Bayes_Deep.pdf

6 Sampling high-dimensional posterior with a simulation based prior

Orateur: Benjamin Remy (CEA Paris-Saclay)

astrodeep_remy.pdf

Slides

Lightning talks: lighting talks session 1

PhD or postdocs present quickly their work

lighting talks

7 Bayesian planetary numerical ephemerides B-INPOP with MCMC

Orateur: Vincenzo Mariani (Observatoire de la Côte d'Azur, GeoAzur)

8 Inferring Cosmological parameters using Normalizing Flows and Gravitational Waves (CosmoFlow)

Orateur: Federico Stachurski (University of Glasgow)

9 Search for ultra-fast radio bursts

Orateur: Kristen Lackeos (Max Planck Institute for Radio Astronomy)

10 Deep Learning Techniques for Time Series Analysis in the context of Gravitational Waves Detection

Orateur: Vlad-Andrei Basceanu (Institute of Space Science, Romania)

10:45 Coffee break

Lightning talks: lighting talks session 2

PhD or postdocs present quickly their work

lighting talks

11 There's no difference: Convolutional Neural Networks for transient detection without template subtraction

Orateur: Tatiana Acero Cuellar (University of Delaware)

12 SNAD miner: Finding Missed Transient Events in ZTF DR4

We report the automatic detection of 11 transients (7 possible supernovae and 4 active galactic nuclei candidates) within the Zwicky Transient Facility fourth data release (ZTF DR4), all of them observed in 2018 and absent from public catalogs. Among these, three were not part of the ZTF alert stream. Our transient mining strategy employs 41 physically motivated features extracted from both real light curves and four simulated light curve models (SN Ia, SN II, TDE, SLSN-I). These features are input to a k-D tree algorithm, from which we calculate the 15 nearest neighbors. After pre-processing and selection cuts, our dataset contained approximately a million objects among which we visually inspected the 105 closest neighbors from seven of our brightest, most well-sampled simulations, comprising 89 unique ZTF DR4 sources. Our result illustrates the potential of coherently incorporating domain knowledge and automatic learning algorithms, which is one of the guiding principles directing the SNAD team. It also demonstrates that the ZTF DR is a suitable testing ground for data mining algorithms aiming to prepare for the next generation of astronomical data.

Orateur: Konstantin Malanchev (University of Illinois at Urbana-Champaign)

Time Domain Astrophysics

13 SNAD: anomaly detection for large scale time-domain astronomy

Orateur: Konstantin Malanchev (University of Illinois at Urbana-Champaign)

snad.key

snad.pdf

snad_w_skipped.pdf

14 A Convolutional Neural Network Approach to Classifying Supernova Time-Series Photometry

Orateur: Helen Qu (University of Pennsylvania)

scone ml talk.key

12:05 Lunch on your own : see booklet infos for advice & suggestions

13:45 Discussions around coffee & tea + poster discussions

Time Domain Astrophysics

15 Bayesian DL methods in extragalactic, time-domain studies

Orateur: Prof. Victoria Ashley Villar (Penn State university)

Bayesian deep learning methods in time-domain studies

16 Searching for changing-state AGNs in massive datasets with anomaly detection

Orateur: Paula Sanchez Saez (European Southern Observatory (ESO))

SanchezSaez_presentation.pdf

17 Adapting to big data: active anomaly detection on ZTF

Orateur: Emille Ishida (Universit ́e Clermont Auvergne, CNRS/IN2P3, LPC)

Adapting to big data: active anomaly detection on ZTF

15:55 Coffee break

Round table: Inference & uncertainties

Président de session: Francois Lanusse ({CNRS}UMR7158)

18 ML inference & uncertainties

Chair : François Lanusse - AIM/CEA/CNRS/Paris-Saclay University/Paris Cite University

Panel :
Yvonne Becherini - Université de Paris, APC, diiP
Emille Ishida - CNRS/LPC-Clermont
Colin Carroll - Google
Benjamin Wandelt - Institut d'Astrophysique de Paris

mercredi 22 juin

08:30 Discussions around coffee & tea + poster discussions

ML Methodology: Simulation based Inference, tools and methods

19 Scalable Bayesian Workflows in JAX

Orateur: Colin Carroll (Google)

slides

20 Fast Inference of Star Formation Histories of Galaxies with Spectra and Simulation-Based Inference (SBI)

Orateur: Gourav Khullar (Dept of Astronomy and Astrophysics, Kavli Institute for Cosmological Physics (KICP), University of Chicago)

Fast Inference of Galaxy Observations with Simulation-Based Inference

21 HARMONIC: Bayesian model comparison for simulation-based inference

Orateur: Matthew Docherty (UCL)

HARMONIC: Bayesian model comparison for simulation-based inference

10:40 Coffee break

Lightning talks: lighting talks session 3

PhD or postdocs present quickly their work

lighting talks

22 Background Estimation in Fermi Gamma-ray Burst Monitor lightcurves through a Neural Network

The aim of this work is to provide a data-driven approach to estimate a background model for the Gamma-Ray Burst Monitor (GBM) of Fermi satellite. We employ a Neural Network (NN) to estimate each detector background signal given the information of the satellite: position, velocity, direction of the detectors, etc.
The estimated background can be employed into a triggering algorithm to discover significant long/weak events that are not previously detected by other approaches.
We show the potential of the model by estimating the background on GBM data for Gamma-Ray Bursts (GRBs) present in GBM cataloge, the long GRB 190320 and ultra-long GRB 091024.
The proposed approach is straightforwardly generalizable to estimate the background model of other satellites.

Orateur: Dr Riccardo Crupi (Università di Udine)

Crupi_Poster.pdf

Crupi_Slide.pdf

23 A Bayesian Convolutional Neural Network for Robust Galaxy Ellipticity Regression

Orateur: Claire Theobald (LORIA)

24 Improvement of the statistical model of a photoz bayesian estimator hybridizing a SED template fitting and a gaussian process

Orateur: Joseph Chevalier (IJCLab)

25 Applying Likelihood-Free Inference to LISA parameter estimation: a project

Orateur: Ivan Martin Vilchez (Institute of Space Science (ICE, CSIC and IEEC))

26 Complete inference for binary-black hole gravitational wave data analysis

When merging, two black holes spiral each other before coalescing into a bigger compact object, distorting the fabric of space-time and emitting gravitational waves. These can be detected and analyzed. Most of the analysis techniques rely on computationally-expensive Bayesian inference software. Recently, machine-learning-based techniques have emerged. These have the advantage that the computational burden is moved upstream; the model needs to be trained once in a more expensive step but is then able to produce samples very rapidly. That way, the analysis of gravitational wave data can be done faster during the observation periods. Since the detection rate increases each time the detectors are upgraded, it means that it will be more and more complicated to keep up the pace to analyze the data. Therefore, fast techniques such as machine-learning-based ones are very interesting. In addition, it is desirable to extract as much information as possible out of the observed events. This requires accounting for the lower mass binary black holes but also probing the higher-order mode content present in the signal. To analyze the entire possible spectrum of binary black hole signals and to extract as much information as possible, we present a deep-learning-based algorithm able to analyze merging compact objects down to a chirp mass of 3 solar masses. Using this method would facilitate the analysis of the increasing amount of gravitational wave events to analyze.

Orateur: Alex Kolmus

ML Methodology: Simulation based Inference, tools and methods

27 Enhancing gravitational-wave population inference with deep learning

Orateur: Matthew Mould (University of Birmingham)

MattewMouldBDL

28 Neural Posterior Estimation with Differentiable Simulators

Orateur: Justine Zeghal (APC)

NPE with Differentiable Simulators

29 Amortized variational inference for supernovae light curves

Orateur: Alexis Sánchez (Universidad de Concepción)

(AVI) for Type Ia Supernova Light Curves, BDL.pdf

30 Self-Supervised Machine Learning; Detecting Galaxy Tidal Features in HSC-SSP

Orateur: Alice Desmons (UNSW Sydney)

Desmons_Alice_methods4_slides.pptx

Merger_sim.mp4

12:40 Lunch on your own : see booklet infos for advice & suggestions

14:00 Discussions around coffee & tea + poster discussions

Gravitational Waves

31 Simulation-based inference for gravitational waves

Orateur: Stephen Green (Albert Einstein Institute Potsdam)

apc-2022-b.pdf

32 Fast Parameter Estimation for Massive Black Hole Binaries with Normalising Flows

Orateur: Natalia Korsakova (APC)

Korsakova_flows_MBHB_APC_20220622.pdf

Poster session: posters and a cup of tea

19:00 Cocktail - La Barge

jeudi 23 juin

08:30 Discussions around coffee & tea + poster discussions

Cosmology

33 Artificial Intelligence: a game-changer for large scale structure cosmology

In large scale structure cosmology, the information about the cosmological parameters governing the evolution of the universe is contained in the complex and rich structure of dark matter density field.
To date, this information was probed using simple human-designed statistics, such as the 2-pt functions, which are not guaranteed or expected to capture the full information content of the LSS maps.
Recently, multiple AI-based methods have been proposed to work with this highly complex data: both for parameter inference and to aid the generation of simulations.
In this talk I will review the progress of practical appliacations of AI to the LSS inference and modelling and highlight with the focus on areas in which AI can be a "game-changer".
I will discuss the recent applications of AI to weak lensing analysis, probe combination, mass map emulators and the creation multi-field simulations.
I will present upcoming simulation sets available to the community that can be used to further advance these techniques.

Orateur: Dr Tomasz Kacprzak (ETH Zurich)

tomaszk_astrodeep_220623.pdf

34 The Extended LSST Astronomical Time Series Classification Challenge (ELAsTiCC)

Orateur: Gautham Narayan (University of Illinois at Urbana-Champaign)

Narayan_ELAsTiCC_BDL_20220623.pdf

35 Hybrid Physical-Neural ODEs for fast differentiable N-body Simulations

Orateur: Denise Lanzieri (Cosmostat, CEA Paris-Saclay)

Hybrid Physical-Neural ODEs for fast differentiable N-body Simulations

10:40 Coffee break

Cosmology

36 Unraveling the role of cosmic velocity field in dark matter halo mass function using deep learning

Orateurs: Erfan Abbasgholinezhad (Sharif Uniersity of Technology), Saba Etezad Razavi (Sharif University of Technology)

BDL 23 June presentation.pdf

37 Graph-Convolutional Neural Networks for large-scale structure clustering

Orateur: Farida Farsian (Bologna University)

Farsian_talk (1).pdf

38 Maximum-A-posteriori estimate with Deep generative NEtworks for Source Separation (MADNESS)

Orateur: Biswajit Biswas (APC)

slides

39 Interpreting non-Gaussian posterior distributions of cosmological parameters with normalizing flows

Orateur: Cyrille Doux (Laboratoire de Physique Subatomique et de Cosmologie)

20220623 BDL 2.pdf

12:30 Lunch on your own : see booklet infos for advice & suggestions

14:00 Discussions around coffee & tea + poster discussions

Ethics of large scale machine learning

40 Keynote

Orateur: Prof. Jean-Gabriel Ganascia (Sorbonne Université)

Ethics_of_Large_Scale_ML_JGGanascia.pptx

41 Goodness-of-fit Evaluation: Toward Trustworthy Probabilistic Machine Learning

Orateur: Arya Farahi (University of Texas at Austin)

BayesianDeepLearning(Final-PDF).pdf

15:50 Coffee break

Cosmology: remote talks

42 Deblending Galaxies with Generative Adversarial Networks

Orateur: Shoubaneh Hemmati (shemmati@caltech.edu)

astrodeep_Hemmati.pdf

43 Galaxies and Halos on Graph Neural Networks: Deep Generative Modeling Scalar and Vector Quantities for Intrinsic Alignment

Orateur: Yesukhei Jagvaral

online slides

44 COSMOPOWER: Deep Learning - accelerated Bayesian inference from next-generation cosmological surveys

Next-generation Large-Scale Structure (LSS) and Cosmic Microwave Background (CMB) surveys will provide us with unprecedented levels of precision in the final constraints on our cosmological model. However, the computational challenges posed by these enormous datasets dangerously hinder their analysis within a Bayesian framework for rigorous uncertainty propagation – a condition necessary to ensure correspondingly high levels of accuracy in the final constraints.

In my talk I will present COSMOPOWER, an open-source Python framework for Deep Learning accelerated Bayesian inference from next-generation CMB and LSS surveys. COSMOPOWER provides orders-of-magnitude acceleration to the inference pipeline by training Deep Learning emulators of matter and CMB power spectra. I will show how these emulators meet the accuracy requirements for application to both currently available cosmological data, such as from the Kilo-Degree Survey (KiDS), as well as to simulated, next-generation data from e.g. a Euclid-like survey. The emulators always recover the fiducial cosmological constraints, while providing a speed-up factor up to Ο(10^4) to the complete inference pipeline. Bayesian parameter contours can thus be recovered in just a few seconds on a common laptop, as opposed to the many hours, days or months of runtime on computer clusters required by standard methods. I will conclude with an outlook on extensions of this software that are currently being developed to extend COSMOPOWER into a fully differentiable library for cosmology.

Orateur: Alessio Spurio Mancini (University College London)

CosmoPower_BDL_Alessio_SpurioMancini.mp4

COSMOPOWER_BDL_Alessio_SpurioMancini.pdf

17:20 break

Round table: ML in the next decades

Président de session: Eric Aubourg (APC)

45 ML in the next decades

Jean-Gabriel Ganascia - Sorbonne Université
Laurent Daudet - LightOn
Anja Butter - IPT Heidelberg

vendredi 24 juin

08:30 Discussions around coffee & tea

ML Tools: Effective scan and while_loop in TF and JAX

Président de session: Junpeng Lao (Google Zurich)

10:30 Coffee break

Conclusion & discussion session

Président de session: Victoria Ashley Villar

slides

12:00 Getting lunch to go : see booklet infos for advice & suggestions

46 Breakout session over lunch