Large language models (LLMs) are used by over a billion people globally, most often to assist with writing. In this work, we demonstrate that LLMs not only alter the voice and tone of human writing, b...

Many real-world processes can be represented as compositions of functions along a directed acyclic graph (DAG). In causal modelling, these correspond to the underlying mechanisms; in engineering, to m...

I recently learned that Springer Nature is about to make the reference journal Statistics and Computing, where I published close to twenty papers over the years, fully “open access”, wh…

State-space models (SSMs) are a broad class of probabilistic models for dynamical systems with many applications in engineering and science. Bayesian filtering is analytically tractable only in the li...

Schedule and book of abstracts

Bayesian filtering is a well-known problem that aims to estimate plausible states of a dynamical system from observations. Among existing approaches to solve this problem, particle filters are theoret...

Bayesian experimental design (BED) provides a principled framework for optimizing data collection by choosing experiments that are maximally informative about unknown parameters. However, existing met...

Squared tensor networks (TNs) and their extension as computational graphs--squared circuits--have been used as expressive distribution estimators, yet supporting closed-form marginalization. However, ...

Many methods across applied mathematics aim at constructing specific parametric probability distributions. Examples of these tasks include evolution strategies or simulated annealing for black-box global optimization, Monte Carlo methods based on adaptive importance sampling, and variational inference algorithms in machine learning. The construction of such distributions can often be formulated as the minimization of a statistical divergence. However, these divergence-minimization problems are challenging because of the following reasons. First, the specific geometry of the considered set of parametric probability distributions needs to be taken into account. Second, efficient evaluations of statistical divergences come with important noise. Third, divergence-minimization problems are generally non-convex. Because of these difficulties, standard methods may fail to converge to good solutions. We tackle these challenges in this thesis.First, we show that evolution strategies, which are sampling-based algorithms for black-box global optimization problems, can be analysed through the lens of divergence-minimization problems. Our approach allows to establish and quantify the improvement brought at each iteration of the algorithms. We show that existing methods fit within our framework, yielding a new approach for their analysis. We also establish improvement results for two novel algorithms, one related with mixture models, and another one using heavy-tailed parametric probability distributions.Second, we consider the minimization of a regularized Rényi divergence over an exponential family. We propose to solve this problem with a stochastic Bregman proximal-gradient algorithm, with biased gradient estimator. By leveraging the geometry of the exponential family, we prove strong convergence guarantees for our algorithm, with proof techniques that are of interest beyond the considered problem. We then extend this algorithm to propose an adaptive simulated annealing algorithm with solid theoretical understanding. We show through a rigorous benchmarking that our algorithm outperforms similar non-adaptive algorithms.Finally, we go beyond exponential families and look at variational inference problems over lambda-exponential families. Using generalized convexity tools, we give new sufficient optimality conditions for these problems, which generalize existing similar results for the exponential family. For the resolution of these problems, we propose novel proximal-like algorithms that exploit the geometry underlying the lambda-exponential family. These results are especially useful for heavy-tailed distributions. We then leverage our results to propose an adaptive importance sampling algorithm to cover these cases. We show that our algorithm is able to learn Student distributions that capture the location, scale, and tail behaviour of target distributions, both in heavy-tailed and light-tailed cases.

Diffusion models offer a robust framework for sampling from unnormalized probability densities, which requires accurately estimating the score of the noise-perturbed target distribution. While the sta...

The ELLIS mission is to create a diverse European network that promotes research excellence and advances breakthroughs in AI, as well as a pan-European PhD program to educate the next generation of AI...