Alex Shtoff @alexshtf.bsky.social · 01/10/2026Good luck in your next endaevors, whatever they are! 010
Alex Shtoff @alexshtf.bsky.social · 23/08/2026AI disclaimer: manuscript written by yours truly, AI assisted in looking up canonical references and related work for literature review; code was heavily AI written and reviewed by yours truly. 000
Alex Shtoff @alexshtf.bsky.social · 23/08/2026I develop the math, give a practical initialization and training recipe, and test the model family on synthetic and real data. This grew out of a question that kept bothering me while working at Yahoo: are there "simple" models that can be scalable, interpretable, and controllable at the same time? 100
Alex Shtoff @alexshtf.bsky.social · 23/08/2026Understanding the above-mentioned nice one-liner began as a series of blog posts, and gradually evolved into this preprint: How expressive does the model become as the matrices grow? What can we read directly from the learned matrices? Which shapes can be guaranteed by construction? 100
Alex Shtoff @alexshtf.bsky.social · 23/08/2026What if a neuron used symmetric matrices as weights - and an eigenvalue as its non-linearity? In a new preprint "The Spectral Neuron", I study models of the form: 𝑓(𝒙) = 𝛌ₖ(𝐀₀ + 𝚺ᵢ 𝑥ᵢ𝐀ᵢ). Manuscript: arxiv.org/abs/2608.08003 Code: github.com/alexshtf/spe... 120
Alex Shtoff @alexshtf.bsky.social · 23/08/2026Just watched another neat interview on the subject to podcast with @primaldual.bsky.social : youtube.com/watch?v=cnN6... Erling's company, MOSEK, is a practical realization of convex optimization and the Just Works⁽™⁾ idea - you give it a problem, and it gives you the solution. No babysitting.youtube.com 010
Alex Shtoff @alexshtf.bsky.social · 18/05/2026The calculus of M-estimation by Stefanski and Boos? 110
Alex Shtoff @alexshtf.bsky.social · 04/02/2026A small library that packages code from my recent blog posts to simplify PyTorch experiments on small datasets and small models that fit into (GPU) memory. Just `pip install fitstream` code: github.com/alexshtf/fit... docs: fitstream.readthedocs.io/en/stable/ 000
Alex Shtoff @alexshtf.bsky.social · 04/01/2026I started exploring the idea of using matrix eigenvalues as the "nonlinearity" in models, and wrote a second post in the series where I explore the scaling, robustness and interpretability properties of this kind of models. The key - feature matrix spectral norms. alexshtf.github.io/2026/01/01/S...alexshtf.github.ioRobustness, interpretability, and scaling of eigenvalue modelsWe discuss mathematical properties that relate to the robustness and interpretability of eigenvalues as models, and demonstrate those by training on a tabular dataset. We obtain a family that improves... 000
Alex Shtoff @alexshtf.bsky.social · 27/07/20254) convergence of deviation from optimality conditions. 110
Alex Shtoff @alexshtf.bsky.social · 26/06/2025I heard that polynomials are the (complex) root of all evil. 010
Reposted by Alex ShtoffClément Canonne @ccanonne.github.io · 06/05/2025Nicely written blog post by David Eppstein on the Boyer–Moore (deterministic) streaming algorithm to find a majority element in a stream, and its extensions, first to the turnstile model, and then to frequency estimation (Misra–Gries). 11011110.github.io/blog/2025/05... via @theory.report11011110.github.ioTurnstile majorityA famous algorithm of Boyer and Moore for the majority problem finds a majority element in a stream of elements while storing only two values, a single tenta... 1183
Reposted by Alex ShtoffSamuel Vaiter @samuelvaiter.com · 01/05/2025The Matrix Mortality Problem asks if a given set of square matrices can multiply to the zero matrix after a finite sequence of multiplications of elements. It is is undecidable for matrices of size 3x3 or larger. buff.ly/lLmvvlo 163
Alex Shtoff @alexshtf.bsky.social · 26/04/2025Attending #ICLR2025? Visit our poster! A stochastic approach to the subset selection problem via mirror descent. Today, 3pm, poster #336. 000
Alex Shtoff @alexshtf.bsky.social · 19/04/2025Don't care who uses meta services. Don't like when people invent imaginary threats to their privacy and spread them. Don't want to use - dont use. Want to opt out and use - dont be afraid they won't comply. Meta is afraid of the legal and reputational consequences. That's my opinion. 100
Alex Shtoff @alexshtf.bsky.social · 19/04/2025Used to work for Yahoo. Not a giant like Meta, but also use plenty of user data to make money. Not complying with regulation was always a big no no. These companies are very afraid of the legal and reputational consequences. So I wouldn't be afraid they won't comply. 100
Alex Shtoff @alexshtf.bsky.social · 18/04/2025The phenomenal paper "epigraphical analysis" by Attouch and Wets was the basis for my Ph.D thesis. It was fun digging deep into epi-convergence. 020
Alex Shtoff @alexshtf.bsky.social · 12/04/2025Yes, I understand. They should cite and criticize it. 100
Alex Shtoff @alexshtf.bsky.social · 12/04/2025Why simply not cite the directly relevant prior work? 100
Alex Shtoff @alexshtf.bsky.social · 12/04/2025A question to the #math people here. For differential equations there are spectral methods that find approximate solutions in the span of orthogonal bases. Is there a variant for difference equations, and bases of sequences? A good tutorial maybe? 000
Reposted by Alex ShtoffSamuel Vaiter @samuelvaiter.com · 01/04/2025The Tarski-Seidenberg theorem in logical form states that the set of first-order formulas over the real numbers is closed under quantifier elimination. This means any formula with quantifiers can be converted into an equivalent quantifier-free formula. perso.univ-rennes1.fr/michel.coste... 062
Alex Shtoff @alexshtf.bsky.social · 31/03/2025🚨New post🚨 @beenwrekt.bsky.social recently started a bit of noise with his post about nonexistence of overfitting, but he has a point. In this post we explore it using simple polynomial curve fitting, *without regularization*, using another interesting basis. alexshtf.github.io/2025/03/27/F... 010
Alex Shtoff @alexshtf.bsky.social · 28/03/2025What makes it a method for "fine-tuning LLMs" rather than a method for fine-tuning any neural network in general? 100
Alex Shtoff @alexshtf.bsky.social · 24/03/2025Is it true that log(1+exp(x)) is the infimum of the quadratic upper bound over a? If so - it also has interesting consequences. 000
Alex Shtoff @alexshtf.bsky.social · 12/03/2025Or maybe there's cultural difference of the black people, who may be more afraid of not returning a loan and may do extreme things, such as using the last of their savings, to return it. This paper seems to focus too much on estimation, and ignores the complexities of modeling. 010
Alex Shtoff @alexshtf.bsky.social · 15/02/2025Models reflect training data. Training data reflects people. Models are just fancy autocomplete. People have free will, and are not fancy autocomplete of what a model has showed them. 000
Alex Shtoff @alexshtf.bsky.social · 14/02/2025The numpy function there doesn't use SGD. To the best of my knowledge, it uses QR decomposition. Anyway, things get interesting when the degree becomes 200 :) 000
Alex Shtoff @alexshtf.bsky.social · 14/02/2025Linear regression with Legendre polynomials: colab.research.google.com/drive/1phA7N... Inspired by Ben's post about nonexistent overfitting, to convince my coworkers.colab.research.google.comGoogle Colab 130
Alex Shtoff @alexshtf.bsky.social · 10/02/2025But, somehow, there is a "uniform prior" over the integers, according to some people I met :) 110
Alex Shtoff @alexshtf.bsky.social · 07/02/2025People underappreciate work that "just works" with current software stacks. Great paper! 110
Alex Shtoff @alexshtf.bsky.social · 01/02/2025For example, spectral methods are good at solving problems where it yields a linear system in the coefficients. 010
Alex Shtoff @alexshtf.bsky.social · 01/02/2025Isn't it like this in, say, numerical differential equations? 110
Alex Shtoff @alexshtf.bsky.social · 28/01/2025There are those two well-known paper: pubsonline.informs.org/doi/abs/10.1... arxiv.org/abs/1610.05708arxiv.orgRelatively-Smooth Convex Optimization by First-Order Methods, and ApplicationsThe usual approach to developing and analyzing first-order methods for smooth convex optimization assumes that the gradient of the objective function is uniformly smooth with some Lipschitz constant $... 120
Alex Shtoff @alexshtf.bsky.social · 26/01/2025Soon we will reinvent A*, and call it "optimal sequential reasoning". 010
Alex Shtoff @alexshtf.bsky.social · 24/01/2025It's not only in AI. It's everywhere. Benchmarks are proxies, and as proxies, they're often exploited to perform well in the benchmark without really trying to be good at what the benchmark aims to measure. Teacher evaluation in schools, employee evaluations at work, etc... are no different. 020
Alex Shtoff @alexshtf.bsky.social · 23/01/2025Reminds me of this slide from a phenomenal tutorial by Prof. Aaditya Ramdas. 000
Alex Shtoff @alexshtf.bsky.social · 20/01/2025Are you referring to Trump's presidency or the upcoming ICLR decisions? :) 010
Reposted by Alex ShtoffTMLR Published Papers @tmlr-pub.bsky.social · 14/01/2025On the Detection of Reviewer-Author Collusion Rings From Paper Bidding Steven Jecmen, Nihar B Shah, Fei Fang, Leman Akoglu Action editor: Laurent Charlin openreview.net/forum?id=o58uy91V2V #collusion #colluders #fraud 032
Reposted by Alex ShtoffTMLR Published Papers @tmlr-pub.bsky.social · 11/01/2025Function Basis Encoding of Numerical Features in Factorization Machines Alex Shtoff, Elie Abboud, Rotem Stram, Oren Somekh Action editor: Andriy Mnih openreview.net/forum?id=M4222IBHsh #factorization #feature #features 011
Alex Shtoff @alexshtf.bsky.social · 11/01/2025Fired: predictions that the world as we know it would end soon, based on the vibes from AI "influencers". 000
Reposted by Alex ShtoffCarola Doerr @caroladoerr.bsky.social · 07/01/2025Videos of the CNRS optimization conference now online (in French): - Claire Mathieu : www.youtube.com/watch?v=_ZXZ... - Gabriel Peyré : www.youtube.com/watch?v=vQOF... - Jérôme Bolte : www.youtube.com/watch?v=tjkg... - Axel Parmentier : www.youtube.com/watch?v=DohO... Enjoy 🙂youtube.com 0409
Alex Shtoff @alexshtf.bsky.social · 04/01/2025How do we fix this? Can we actually fight this massive abuse somehow? It's important, both for humanity and for us, to invest research resources into this. 000
