iiinova — where your digital self keeps evolving

Ethan Epperly @ethanepperly.bsky.social · 05/08/2025

New blog post up about the amazingly useful Gaussian integration by parts formula! As an application, we use it to analyze power iteration from a random start www.ethanepperly.com/index.php/20...

Gaussian integration by parts. Let z be a standard Gaussian random variable. Then E[zf(z)] = E[f'(z)].

000

Ethan Epperly @ethanepperly.bsky.social · 16/06/2025

New blog post up about the randomized Kaczmarz algorithm. The classic RK algorithms samples rows according to their squared norms, but what happens if you sample them uniformly? The answer surprised me: Uniform sampling is often just as good or even better www.ethanepperly.com/index.php/20...

120

Ethan Epperly @ethanepperly.bsky.social · 25/02/2025

Ack! Typesetting glitch. It was meant to be diag(x) A diag(x) M

$We start by computing $x^\top (A\circ M)x$: $$x^\top (A\circ M)x = \sum_{i,j=1}^n x_i (A\circ M)_{ij} x_j = \sum_{i,j=1}^n x_i A_{ij} M_{ij} x_j.$$Now, we may rearrange the sum, use symmetry of $M$, and repackage it as a trace $$x^\top (A\circ M)x = \sum_{i,j=1}^n x_i A_{ij} x_j M_{ji} = \tr(\operatorname{diag}(x) A \operatorname{diag}(x) M).$$This the trace formula for quadratic forms in the Schur product.$

110

Ethan Epperly @ethanepperly.bsky.social · 25/02/2025

New blog post with four proofs of the Schur product theorem. Do you know a fifth? www.ethanepperly.com/index.php/20...

$Proof of the Schur product theorem: The Kronecker product $A\otimes M$ of two psd matrices is psd. The entrywise product $A\circ M$ is a principal submatrix of $A\otimes M$: $$A\circ M = ((A\otimes M)_{(i+n(i-1))(i+n(i-1))} : i = 1,\ldots,n).$$All principal submatrices of a psd matrix are psd, so $A\circ M$ is psd.$

110

Ethan Epperly @ethanepperly.bsky.social · 02/12/2024

New paper out with Gil Goldshlager and Rob Webber! In it, we show that *tail averaging* can be used to improve the accuracy of the randomized Kaczmarz method for solving least-squares problems. The resulting method, TARK, outcompetes other row-access methods for least squares

Error for different randomized Kaczmarz methods applied to a least-squares problem. Tail-averaged randomized Kaczmarz (TARK) outcompetes the existing methods

120

Ethan Epperly @ethanepperly.bsky.social · 21/11/2024

New blog post up presenting some beautiful *exact formulas* for sketched least squares with a Gaussian embedding. These beautiful formulas appear to have only been published as recently as 2020; see post for details! www.ethanepperly.com/index.php/20...

130