Alex Shtoff @alexshtf.bsky.social · 23/08/2026What if a neuron used symmetric matrices as weights - and an eigenvalue as its non-linearity? In a new preprint "The Spectral Neuron", I study models of the form: 𝑓(𝒙) = 𝛌ₖ(𝐀₀ + 𝚺ᵢ 𝑥ᵢ𝐀ᵢ). Manuscript: arxiv.org/abs/2608.08003 Code: github.com/alexshtf/spe... 120
Alex Shtoff @alexshtf.bsky.social · 23/08/2026Just watched another neat interview on the subject to podcast with @primaldual.bsky.social : youtube.com/watch?v=cnN6... Erling's company, MOSEK, is a practical realization of convex optimization and the Just Works⁽™⁾ idea - you give it a problem, and it gives you the solution. No babysitting.youtube.com 010
Alex Shtoff @alexshtf.bsky.social · 04/02/2026A small library that packages code from my recent blog posts to simplify PyTorch experiments on small datasets and small models that fit into (GPU) memory. Just `pip install fitstream` code: github.com/alexshtf/fit... docs: fitstream.readthedocs.io/en/stable/ 000
Alex Shtoff @alexshtf.bsky.social · 04/01/2026I started exploring the idea of using matrix eigenvalues as the "nonlinearity" in models, and wrote a second post in the series where I explore the scaling, robustness and interpretability properties of this kind of models. The key - feature matrix spectral norms. alexshtf.github.io/2026/01/01/S...alexshtf.github.ioRobustness, interpretability, and scaling of eigenvalue modelsWe discuss mathematical properties that relate to the robustness and interpretability of eigenvalues as models, and demonstrate those by training on a tabular dataset. We obtain a family that improves... 000
Reposted by Alex ShtoffClément Canonne @ccanonne.github.io · 06/05/2025Nicely written blog post by David Eppstein on the Boyer–Moore (deterministic) streaming algorithm to find a majority element in a stream, and its extensions, first to the turnstile model, and then to frequency estimation (Misra–Gries). 11011110.github.io/blog/2025/05... via @theory.report11011110.github.ioTurnstile majorityA famous algorithm of Boyer and Moore for the majority problem finds a majority element in a stream of elements while storing only two values, a single tenta... 1183
Reposted by Alex ShtoffSamuel Vaiter @samuelvaiter.com · 01/05/2025The Matrix Mortality Problem asks if a given set of square matrices can multiply to the zero matrix after a finite sequence of multiplications of elements. It is is undecidable for matrices of size 3x3 or larger. buff.ly/lLmvvlo 163
Alex Shtoff @alexshtf.bsky.social · 26/04/2025Attending #ICLR2025? Visit our poster! A stochastic approach to the subset selection problem via mirror descent. Today, 3pm, poster #336. 000
Alex Shtoff @alexshtf.bsky.social · 12/04/2025A question to the #math people here. For differential equations there are spectral methods that find approximate solutions in the span of orthogonal bases. Is there a variant for difference equations, and bases of sequences? A good tutorial maybe? 000
Reposted by Alex ShtoffSamuel Vaiter @samuelvaiter.com · 01/04/2025The Tarski-Seidenberg theorem in logical form states that the set of first-order formulas over the real numbers is closed under quantifier elimination. This means any formula with quantifiers can be converted into an equivalent quantifier-free formula. perso.univ-rennes1.fr/michel.coste... 062
Alex Shtoff @alexshtf.bsky.social · 31/03/2025🚨New post🚨 @beenwrekt.bsky.social recently started a bit of noise with his post about nonexistence of overfitting, but he has a point. In this post we explore it using simple polynomial curve fitting, *without regularization*, using another interesting basis. alexshtf.github.io/2025/03/27/F... 010
Reposted by Alex ShtoffTMLR Published Papers @tmlr-pub.bsky.social · 14/01/2025On the Detection of Reviewer-Author Collusion Rings From Paper Bidding Steven Jecmen, Nihar B Shah, Fei Fang, Leman Akoglu Action editor: Laurent Charlin openreview.net/forum?id=o58uy91V2V #collusion #colluders #fraud 032
Reposted by Alex ShtoffTMLR Published Papers @tmlr-pub.bsky.social · 11/01/2025Function Basis Encoding of Numerical Features in Factorization Machines Alex Shtoff, Elie Abboud, Rotem Stram, Oren Somekh Action editor: Andriy Mnih openreview.net/forum?id=M4222IBHsh #factorization #feature #features 011
Reposted by Alex ShtoffCarola Doerr @caroladoerr.bsky.social · 07/01/2025Videos of the CNRS optimization conference now online (in French): - Claire Mathieu : www.youtube.com/watch?v=_ZXZ... - Gabriel Peyré : www.youtube.com/watch?v=vQOF... - Jérôme Bolte : www.youtube.com/watch?v=tjkg... - Axel Parmentier : www.youtube.com/watch?v=DohO... Enjoy 🙂youtube.com 0409
Alex Shtoff @alexshtf.bsky.social · 04/01/2025Fellow AI researchers. Please watch this video. Freya raises a valid concern about the sheer abuse of GenAI on the web, and the damage it does. 100
Alex Shtoff @alexshtf.bsky.social · 04/01/2025[1/4] When working on ads at Yahoo, we had several 'ad hoc' solutions for various problems, and one of them was exponential moving average (EMA) of observations y₁,y₂...: xᵢ₊₁=(1 - α)xᵢ+αyᵢ One of the most overlooked facts is that it is actually online gradient descent! 100
Alex Shtoff @alexshtf.bsky.social · 01/01/2025🚀 New Paper 🚀 This post is about our recent TMLR paper, "Function Basis Encoding of Numerical Features in Factorization Machines", by Alex Shtoff, Elie Abboud, Rotem Stram, and Oren Somekh. Paper: openreview.net/forum?id=M42... Code: github.com/alexshtf/con... 100
Reposted by Alex ShtoffTMLR Published Papers @tmlr-pub.bsky.social · 26/12/2024Your Classifier Can Be Secretly a Likelihood-Based OOD Detector Jirayu Burapacheep, Yixuan Li Action editor: Changjian Shui openreview.net/forum?id=FmA1JPWBM8 #classifiers #classifier #classification 082
Alex Shtoff @alexshtf.bsky.social · 15/12/2024One of the overlooked properties of the proximal operator is the formula for composing with a semi-orthogonal matrix: h(x) = p(A x + b), with A Aᵀ = α I It is [1]: proxₜₕ(x) = x + t⁻¹ Aᵀ(proxₜₚ(A x) - A x) [1] Combettes, Wajs. Signal Recovery by Proximal Forward-Backward Splitting. 100
Alex Shtoff @alexshtf.bsky.social · 13/12/2024Help me here #ML/#LLM X please. I'm pretty sure someone already thought of the extremely simple idea of augmenting each attention layer with additional learnable auxiliary memory in the form of embeddings. Could you point me to papers? 100
Alex Shtoff @alexshtf.bsky.social · 12/12/2024M-Estimation is all you need Change my mind :) www.jstor.org/stable/3087324jstor.org The Calculus of M-Estimation on JSTOR Leonard A. Stefanski, Dennis D. Boos, The Calculus of M-Estimation, The American Statistician, Vol. 56, No. 1 (Feb., 2002), pp. 29-38 000
Reposted by Alex ShtoffGergely Neu @neu-rips.bsky.social · 04/12/2024exciting new work by my truly brilliant postdoc Eugenio Clerico on the optimality of coin-betting strategies for mean estimation! for fans of: mean estimation, online learning with log loss, optimal portfolios, hypothesis testing with E-values, etc. dig in: arxiv.org/abs/2412.02640arxiv.orgOn the optimality of coin-betting for mean estimationConfidence sequences are sequences of confidence sets that adapt to incoming data while maintaining validity. Recent advances have introduced an algorithmic formulation for constructing some of the ti... 2418
Alex Shtoff @alexshtf.bsky.social · 01/12/2024I just found out that many people in the industry say that logistic regression (sigmoid + BCE loss) is not a regression algorithm, but a classification algorithm. And that the name "Logistic **Regression**" is wrong... How do you call a model for estimating the conditional mean? 220
Alex Shtoff @alexshtf.bsky.social · 28/11/2024@bsky.app What's the excuse for blocking alpindale? Is ML research considered "trolling the community"? 100
Reposted by Alex ShtoffClément Canonne @ccanonne.github.io · 26/11/2024The PCP theorem, a jewel of theoretical computer science, establishes that any NP statement can be assessed by a randomized verifier who only checks a vanishing fraction of the proof (indeed, a constant # of characters!) This has had incredible impact, most notably on how ML reviews are conducted 715520
Reposted by Alex ShtoffMotonobu Kanagawa @motonobu-kanagawa.bsky.social · 17/11/2024We are organising the First International Conference on Probabilistic Numerics (ProbNum 2025) at EURECOM in southern France in Sep 2025. Topics: AI, ML, Stat, Sim, and Numerics. Reposts very much appreciated! probnum25.github.io 34624
Alex Shtoff @alexshtf.bsky.social · 26/11/2024Everybody likes complaining about ICLR... but after a rebuttal phase, the authors of one of the papers I reviewed addressed my concerns well, revised the paper according to the reviews of the other reviewers, and I increased their score. At least here - the process worked :) 000
Alex Shtoff @alexshtf.bsky.social · 24/11/2024After a short time here, it's time for a short intro thread. I like things on the intersection between opimization, numerical analysis, ML, and software engineering. I have a blog, where I write about stuff I like: alexshtf.github.io You're welcome to add me to starter packs as you see fit.alexshtf.github.ioAlex ShtoffBlog on optimization, machine learning, and software development. 110
Reposted by Alex ShtoffRobert "The Bobby Yaga" McNees @mcnees.bsky.social · 24/11/2024Charles Darwin's "On the Origin of Species by Means of Natural Selection" was published #OTD in 1859. 🧪 (1/n) 1634750
Reposted by Alex ShtoffVictor Zhong @vzhong.bsky.social · 23/11/2024Are you on the job market? Our department at the University of Waterloo is hiring for multiple tenure track positions in CS, please apply here! Feel free to DM me with questions. cs.uwaterloo.ca/tenure-track...cs.uwaterloo.caTenure-track Faculty Positions | Cheriton School of Computer ScienceTenure-track Faculty Positions ad 0216
Alex Shtoff @alexshtf.bsky.social · 21/11/2024Want to try out incrementally train Poisson regression models in your #ML research using proximal-point style methods, so that you don't suffer from `exploding gradients`? Here is a small #Python library for you! github.com/alexshtf/exp...github.comGitHub - alexshtf/exp_prox_pt: Proximal point with exponential lossesProximal point with exponential losses. Contribute to alexshtf/exp_prox_pt development by creating an account on GitHub. 020
Alex Shtoff @alexshtf.bsky.social · 19/11/2024I've heard your first post has to be provocative... So, in machine learning, optimization is all you need. Change my mind! 240