Sign in

Alex Shtoff

@alexshtf.bsky.social
52 followers 98 following 135 posts

Principal scientist @ TII Visit my research blog at alexshtf.github.io

PostsRepliesMedia
Alex Shtoff @alexshtf.bsky.social · 23/08/2026
What if a neuron used symmetric matrices as weights - and an eigenvalue as its non-linearity? In a new preprint "The Spectral Neuron", I study models of the form: 𝑓(𝒙) = 𝛌ₖ(𝐀₀ + 𝚺ᵢ 𝑥ᵢ𝐀ᵢ). Manuscript: arxiv.org/abs/2608.08003 Code: github.com/alexshtf/spe...
120
Alex Shtoff @alexshtf.bsky.social · 23/08/2026
Just watched another neat interview on the subject to podcast with @primaldual.bsky.social : youtube.com/watch?v=cnN6... Erling's company, MOSEK, is a practical realization of convex optimization and the Just Works⁽™⁾ idea - you give it a problem, and it gives you the solution. No babysitting.
youtube.com
010
Alex Shtoff @alexshtf.bsky.social · 04/02/2026
A small library that packages code from my recent blog posts to simplify PyTorch experiments on small datasets and small models that fit into (GPU) memory. Just `pip install fitstream` code: github.com/alexshtf/fit... docs: fitstream.readthedocs.io/en/stable/
000
Alex Shtoff @alexshtf.bsky.social · 04/01/2026
I started exploring the idea of using matrix eigenvalues as the "nonlinearity" in models, and wrote a second post in the series where I explore the scaling, robustness and interpretability properties of this kind of models. The key - feature matrix spectral norms. alexshtf.github.io/2026/01/01/S...
alexshtf.github.io
Robustness, interpretability, and scaling of eigenvalue models
We discuss mathematical properties that relate to the robustness and interpretability of eigenvalues as models, and demonstrate those by training on a tabular dataset. We obtain a family that improves...
000
Reposted by Alex Shtoff
Clément Canonne @ccanonne.github.io · 06/05/2025
Nicely written blog post by David Eppstein on the Boyer–Moore (deterministic) streaming algorithm to find a majority element in a stream, and its extensions, first to the turnstile model, and then to frequency estimation (Misra–Gries). 11011110.github.io/blog/2025/05... via @theory.report
11011110.github.io
Turnstile majority
A famous algorithm of Boyer and Moore for the majority problem finds a majority element in a stream of elements while storing only two values, a single tenta...
1183
Reposted by Alex Shtoff
Samuel Vaiter @samuelvaiter.com · 01/05/2025
The Matrix Mortality Problem asks if a given set of square matrices can multiply to the zero matrix after a finite sequence of multiplications of elements. It is is undecidable for matrices of size 3x3 or larger. buff.ly/lLmvvlo
163
Alex Shtoff @alexshtf.bsky.social · 26/04/2025
Attending #ICLR2025? Visit our poster! A stochastic approach to the subset selection problem via mirror descent. Today, 3pm, poster #336.
000
Alex Shtoff @alexshtf.bsky.social · 12/04/2025
A question to the #math people here. For differential equations there are spectral methods that find approximate solutions in the span of orthogonal bases. Is there a variant for difference equations, and bases of sequences? A good tutorial maybe?
000
Reposted by Alex Shtoff
Samuel Vaiter @samuelvaiter.com · 01/04/2025
The Tarski-Seidenberg theorem in logical form states that the set of first-order formulas over the real numbers is closed under quantifier elimination. This means any formula with quantifiers can be converted into an equivalent quantifier-free formula. perso.univ-rennes1.fr/michel.coste...
062
Alex Shtoff @alexshtf.bsky.social · 31/03/2025
🚨New post🚨 @beenwrekt.bsky.social recently started a bit of noise with his post about nonexistence of overfitting, but he has a point. In this post we explore it using simple polynomial curve fitting, *without regularization*, using another interesting basis. alexshtf.github.io/2025/03/27/F...
010
Reposted by Alex Shtoff
TMLR Published Papers @tmlr-pub.bsky.social · 14/01/2025
On the Detection of Reviewer-Author Collusion Rings From Paper Bidding Steven Jecmen, Nihar B Shah, Fei Fang, Leman Akoglu Action editor: Laurent Charlin openreview.net/forum?id=o58uy91V2V #collusion #colluders #fraud
032
Reposted by Alex Shtoff
TMLR Published Papers @tmlr-pub.bsky.social · 11/01/2025
Function Basis Encoding of Numerical Features in Factorization Machines Alex Shtoff, Elie Abboud, Rotem Stram, Oren Somekh Action editor: Andriy Mnih openreview.net/forum?id=M4222IBHsh #factorization #feature #features
011
Reposted by Alex Shtoff
Carola Doerr @caroladoerr.bsky.social · 07/01/2025
Videos of the CNRS optimization conference now online (in French): - Claire Mathieu : www.youtube.com/watch?v=_ZXZ... - Gabriel Peyré : www.youtube.com/watch?v=vQOF... - Jérôme Bolte : www.youtube.com/watch?v=tjkg... - Axel Parmentier : www.youtube.com/watch?v=DohO... Enjoy 🙂
youtube.com
0409
Alex Shtoff @alexshtf.bsky.social · 04/01/2025
Fellow AI researchers. Please watch this video. Freya raises a valid concern about the sheer abuse of GenAI on the web, and the damage it does.
100
Alex Shtoff @alexshtf.bsky.social · 04/01/2025
[1/4] When working on ads at Yahoo, we had several 'ad hoc' solutions for various problems, and one of them was exponential moving average (EMA) of observations y₁,y₂...: xᵢ₊₁=(1 - α)xᵢ+αyᵢ One of the most overlooked facts is that it is actually online gradient descent!
100
Alex Shtoff @alexshtf.bsky.social · 01/01/2025
🚀 New Paper 🚀 This post is about our recent TMLR paper, "Function Basis Encoding of Numerical Features in Factorization Machines", by Alex Shtoff, Elie Abboud, Rotem Stram, and Oren Somekh. Paper: openreview.net/forum?id=M42... Code: github.com/alexshtf/con...
100
Alex Shtoff @alexshtf.bsky.social · 30/12/2024
ICLR area chairs reviewing papers
000
Reposted by Alex Shtoff
TMLR Published Papers @tmlr-pub.bsky.social · 26/12/2024
Your Classifier Can Be Secretly a Likelihood-Based OOD Detector Jirayu Burapacheep, Yixuan Li Action editor: Changjian Shui openreview.net/forum?id=FmA1JPWBM8 #classifiers #classifier #classification
082
Alex Shtoff @alexshtf.bsky.social · 15/12/2024
One of the overlooked properties of the proximal operator is the formula for composing with a semi-orthogonal matrix: h(x) = p(A x + b), with A Aᵀ = α I It is [1]: proxₜₕ(x) = x + t⁻¹ Aᵀ(proxₜₚ(A x) - A x) [1] Combettes, Wajs. Signal Recovery by Proximal Forward-Backward Splitting.
100
Alex Shtoff @alexshtf.bsky.social · 13/12/2024
Help me here #ML/#LLM X please. I'm pretty sure someone already thought of the extremely simple idea of augmenting each attention layer with additional learnable auxiliary memory in the form of embeddings. Could you point me to papers?
100
Alex Shtoff @alexshtf.bsky.social · 12/12/2024
M-Estimation is all you need Change my mind :) www.jstor.org/stable/3087324
jstor.org
The Calculus of M-Estimation on JSTOR
Leonard A. Stefanski, Dennis D. Boos, The Calculus of M-Estimation, The American Statistician, Vol. 56, No. 1 (Feb., 2002), pp. 29-38
000
Reposted by Alex Shtoff
Gergely Neu @neu-rips.bsky.social · 04/12/2024
exciting new work by my truly brilliant postdoc Eugenio Clerico on the optimality of coin-betting strategies for mean estimation! for fans of: mean estimation, online learning with log loss, optimal portfolios, hypothesis testing with E-values, etc. dig in: arxiv.org/abs/2412.02640
arxiv.org
On the optimality of coin-betting for mean estimation
Confidence sequences are sequences of confidence sets that adapt to incoming data while maintaining validity. Recent advances have introduced an algorithmic formulation for constructing some of the ti...
2418
Alex Shtoff @alexshtf.bsky.social · 01/12/2024
I just found out that many people in the industry say that logistic regression (sigmoid + BCE loss) is not a regression algorithm, but a classification algorithm. And that the name "Logistic **Regression**" is wrong... How do you call a model for estimating the conditional mean?
220
Alex Shtoff @alexshtf.bsky.social · 28/11/2024
@bsky.app What's the excuse for blocking alpindale? Is ML research considered "trolling the community"?
100
Alex Shtoff @alexshtf.bsky.social · 26/11/2024
Beautiful!
000
Reposted by Alex Shtoff
Clément Canonne @ccanonne.github.io · 26/11/2024
The PCP theorem, a jewel of theoretical computer science, establishes that any NP statement can be assessed by a randomized verifier who only checks a vanishing fraction of the proof (indeed, a constant # of characters!) This has had incredible impact, most notably on how ML reviews are conducted
715520
Reposted by Alex Shtoff
Motonobu Kanagawa @motonobu-kanagawa.bsky.social · 17/11/2024
We are organising the First International Conference on Probabilistic Numerics (ProbNum 2025) at EURECOM in southern France in Sep 2025. Topics: AI, ML, Stat, Sim, and Numerics. Reposts very much appreciated! probnum25.github.io
34624
Alex Shtoff @alexshtf.bsky.social · 26/11/2024
Everybody likes complaining about ICLR... but after a rebuttal phase, the authors of one of the papers I reviewed addressed my concerns well, revised the paper according to the reviews of the other reviewers, and I increased their score. At least here - the process worked :)
000
Alex Shtoff @alexshtf.bsky.social · 24/11/2024
After a short time here, it's time for a short intro thread. I like things on the intersection between opimization, numerical analysis, ML, and software engineering. I have a blog, where I write about stuff I like: alexshtf.github.io You're welcome to add me to starter packs as you see fit.
alexshtf.github.io
Alex Shtoff
Blog on optimization, machine learning, and software development.
110
Alex Shtoff @alexshtf.bsky.social · 24/11/2024
Nice!
000
Reposted by Alex Shtoff
Robert "The Bobby Yaga" McNees @mcnees.bsky.social · 24/11/2024
Charles Darwin's "On the Origin of Species by Means of Natural Selection" was published #OTD in 1859. 🧪 (1/n)
Cover page of Darwin’s “On the Origin of Species by Means of Natural Selection.” The paper is a mellow, light brown color from age and acidification.
1634750
Reposted by Alex Shtoff
Victor Zhong @vzhong.bsky.social · 23/11/2024
Are you on the job market? Our department at the University of Waterloo is hiring for multiple tenure track positions in CS, please apply here! Feel free to DM me with questions. cs.uwaterloo.ca/tenure-track...
cs.uwaterloo.ca
Tenure-track Faculty Positions | Cheriton School of Computer Science
Tenure-track Faculty Positions ad
0216
Alex Shtoff @alexshtf.bsky.social · 21/11/2024
Want to try out incrementally train Poisson regression models in your #ML research using proximal-point style methods, so that you don't suffer from `exploding gradients`? Here is a small #Python library for you! github.com/alexshtf/exp...
github.com
GitHub - alexshtf/exp_prox_pt: Proximal point with exponential losses
Proximal point with exponential losses. Contribute to alexshtf/exp_prox_pt development by creating an account on GitHub.
020
Alex Shtoff @alexshtf.bsky.social · 19/11/2024
I've heard your first post has to be provocative... So, in machine learning, optimization is all you need. Change my mind!
240