iiinova — where your digital self keeps evolving

Kacper Wyrwal

@wyrwalkacper.bsky.social

27 followers 13 following 10 posts

I dabble in geometric machine learning. Master's student at the University of Oxford.

Posts Replies Media

Kacper Wyrwal @wyrwalkacper.bsky.social · 24/10/2025

If you're looking for a #PhD in #Geometric or #Probabilistic #MachineLearning, I highly recommend applying! Viacheslav is an excellent mentor, outstanding researcher, and a genuinely great person. Speaking from experience, I can't recommend him enough as a supervisor.

020

Kacper Wyrwal @wyrwalkacper.bsky.social · 13/02/2025

All this is with little to no fine-tuning! Simply initialising hidden layers with a small variance allows our model to use additional layers just when necessary, preventing overfitting in our experiments. See our paper here: arxiv.org/abs/2411.00161. 9/n

020

Kacper Wyrwal @wyrwalkacper.bsky.social · 13/02/2025

Lastly, we demonstrate that residual deep GPs can be faster than Euclidean deep GPs, by projecting Euclidean data to a compact manifold. 8/n

110

Kacper Wyrwal @wyrwalkacper.bsky.social · 13/02/2025

We test our model on the ERA5 dataset – interpolating wind on the globe from a set of points on a satellite trajectory. Our model outperforms baselines, yielding accurate and interpretable uncertainty estimates. An example predictive mean and variance is shown below. 7/n

The posterior mean and uncertainty of a 3-layer residual deep GP trained on ERA5 wind velocity data at an altitude of 0.1 km from July 2010. The mean is shown as black arrows, while the predictive uncertainty is shown using a colour scale from purple (lowest) to yellow (highest).

131

Kacper Wyrwal @wyrwalkacper.bsky.social · 13/02/2025

Our model can also serve as a plug-and-play replacement for shallow manifold GPs in geometry-aware Bayesian optimisation. This can be especially useful for complex target functions, as we demonstrate experimentally. 6/n

100

Kacper Wyrwal @wyrwalkacper.bsky.social · 13/02/2025

With manifold GPs at every layer, we can leverage manifold-specific methods like intrinsic Gaussian vector fields and interdomain inducing variables to improve performance. 5/n

110

Kacper Wyrwal @wyrwalkacper.bsky.social · 13/02/2025

We can build deep GPs by stacking these layers. Each layer learns a translation of inputs, allowing incremental updates of hidden representations – just like the ResNet! In fact, on the Euclidean manifold, we recover the ResNet-inspired deep GP of Salimbeni & @deisenroth.bsky.social. 4/n

110

Kacper Wyrwal @wyrwalkacper.bsky.social · 13/02/2025

Not quite. On general manifolds, points and tangent vectors cannot be identified. We can, however, translate points in the direction of vectors using the exponential map. Thus, we define a manifold-to-manifold GP as a composition of a Gaussian vector field with this map. 3/n

120

Kacper Wyrwal @wyrwalkacper.bsky.social · 13/02/2025

When Euclidean GPs struggle to model irregular functions, stacking them into a deep GPs can help. This works because points and vectors in Euclidean space can be identified, allowing a vector-valued GP’s output to serve as another’s input. But can we do this on manifolds? 2/n

110

Kacper Wyrwal @wyrwalkacper.bsky.social · 13/02/2025

Excited to share our ICLR 2025 oral "Residual Deep Gaussian Processes on Manifolds"! With @vabor112.bsky.social & @arkrause.bsky.social, we introduce manifold-to-manifold GPs that can be composed together, generalising deep GPs to manifolds. Applications include wind prediction & Bayes opt! 1/n

Schematic illustration of a scalar-valued residual deep GP with L hidden layers. The last layer is a scalar-valued GP on the manifold. If it is not present, the model is manifold-valued. If it is replaced with a Gaussian vector field (GVF), the model is a vector field on the manifold.

1379