iiinova — where your digital self keeps evolving

Great question! We have not directly compared. SMC offers a test-time approach to steer off-the-shelf models without additional training, whereas diffusion forcing trains autoregressive models to more effectively sample from the global target. These strategies can actually likely be combined.

030

And check out the following papers, which set the technical landscape for this work to build on: Lew et al (2023): arxiv.org/abs/2306.03081 Loula et al (2025): arxiv.org/abs/2504.13139

Thanks to Ben LeBrun, @vigly.bsky.social bsky.social, @joaoloula.bsky.social sky.social, @drmaciver.bsky.social aciver.bsky.social, Li Du, Jason Eisner, Ryan Cotterell, Vikash Mansinghka, Tim O'Donnell, @alexlew.bsky.social ky.social, @xtimv.bsky.social Paper Link: arxiv.org/abs/2504.05410

Want to use AWRS SMC? Check out the GenLM control library: github.com/genlm/genlm-... GenLM supports not only grammars, but arbitrary programmable constraints from type systems to simulators. If you can write a Python function, you can control your language model!

161

Why does AWRS work? Formal and empirical runtime analyses tell a fascinating story. AWRS scales adaptively with the KL divergence between the conditional and base token-level models. As your LM better understands the constraint, AWRS gets faster. As the LM struggles, AWRS closes the gap.

We tested AWRS SMC on several controlled generation tasks, from text-to-SQL to PDDL goal inference to molecular synthesis. AWRS SMC outperforms baselines by large margins, e.g., see the jump from 3% -> 53% in the goal inference domain with only ~2.5x clock time overhead.

Next, SMC uses the proposed extensions and corresponding weights from AWRS to update importance weights associated with partial sequences (particles). These particles are resampled proportional to their weights, re-allocating computation towards the most promising sequences.

First, AWRS reformulates the token-level inference problem from exact enumeration to adaptive rejection sampling. This process yields equivalently distributed samples at a fraction of the cost. AWRS then estimates and propagates an importance weight alongside these samples.

So, what can we do? AWRS SMC is a hierarchical inference framework based on sequential Monte Carlo using a novel stochastic proposal algorithm. By jointly considering local and global signals, AWRS SMC is both probabilistically sound and sample efficient. How does it work?

Problem B: LCD distorts the distribution. Consider this simple LM over the tokens `a` and `b` with the constraint that “strings must end with `a`”. While the distribution on complete strings favors `ba`, autoregressive sampling will favor `ab`. We don’t want this.

Problem A: Token masking is often slow. Must classify all 100,000+ tokens in the vocab at each step. While regular and context-free grammars support low-overhead solutions using tools like Outlines (dottxtai.bsky.social), open-ended constraint enforcement has been harder.

Approach 2: Locally constrained decoding (LCD). At each step, mask the next-token distribution to prevent violations. Pros: All samples are constraint-satisfying. Cons: A) Masking a large vocabulary is slow. B) LCD distorts the sampled distribution. Example:

Approach 1: Sample-verify/Best-of-N. Draw 𝑁 strings from the LM and use the constraint to rank/filter. Pros: Samples 𝑥 ∝ 𝑃 as 𝑁 grows. Cons: 𝑁 required to get a target sample scales exp(KL[𝑃||𝑄]). For difficult constraints, this becomes infeasible. Example:

Consider a prompted language model 𝑄 (a prior) and a constraint function 𝐶 (a likelihood). Our goal is to sample a string 𝑥 from the conditional distribution 𝑃 = 𝑄(·|𝐶(𝑥)=1) (the target posterior). How do people do this now, and why do current approaches fall short?

João Loula @joaoloula.bsky.social · 25/04/2025

Many LM applications may be formulated as text generation conditional on some (Boolean) constraint. Generate a… - Python program that passes a test suite. - PDDL plan that satisfies a goal. - CoT trajectory that yields a positive reward. The list goes on… How can we efficiently satisfy these? 🧵👇

2136

Reposted by Ben Lipkin

#ICLR2025 Oral How can we control LMs using diverse signals such as static analyses, test cases, and simulations? In our paper “Syntactic and Semantic Control of Large Language Models via Sequential Monte Carlo” (w/ @benlipkin.bsky.social, @alexlew.bsky.social, @xtimv.bsky.social) we:

176

Reposted by Ben Lipkin

Kyle Mahowald @kmahowald.bsky.social · 21/04/2025

I might be able to hire a postdoc for this fall in computational linguistics at UT Austin. Topics in the general LLM + cognitive space (particularly reasoning, chain of thought, LLMs + code) and LLM + linguistic space. If this could be of interest, feel free to get in touch!

05930

Ben Lipkin @benlipkin.bsky.social · 10/04/2025

Big thanks to this awesome team: Ben LeBrun, @postylem.bsky.social, @joaoloula.bsky.social, @drmaciver.bsky.social, Li Du, Jason Eisner, Ryan Cotterell, Vikash Mansinghka, Tim O'Donnell, @alexlew.bsky.social, @xtimv.bsky.social

020

Ben Lipkin @benlipkin.bsky.social · 10/04/2025

New preprint on controlled generation from LMs! I'll be presenting at NENLP tomorrow 12:50-2:00pm Longer thread coming soon :)

1209

Reposted by Ben Lipkin

Greta Tuckute @gretatuckute.bsky.social · 15/12/2024

I defended my PhD at MIT Brain&Cog last week--so much gratitude to my advisor @evfedorenko.bsky.social, as well as my committee @nancykanwisher.bsky.social, @joshhmcdermott.bsky.social and Yoon Kim. Thank you to all my brilliant collaborators and the MIT community. I have loved this journey so much.

5996

Reposted by Ben Lipkin

Alex Lew @alexlew.bsky.social · 08/12/2024

If you're interested in a PhD at the intersection of machine learning and programming languages, consider applying to Yale CS! We're exploring new approaches to building software that draws inferences and makes predictions. See alexlew.net for details & apply at gsas.yale.edu/admissions/ by Dec. 15

Probabilistic and differentiable programming at Yale — fully funded PhD positions starting Fall 2025! Apply by Dec. 15.

Do a PhD at the rich intersection of programming languages and machine learning.

17322

If this sounds useful for your work, feel free to reach out, and I’m happy to explore with you how `decoding` can best support your use cases The goal is for this library to be generally useful for researchers across the LLM landscape 7/7

000

Check out the tutorial to see some examples in action: github.com/benlipkin/de... The library is fully documented: benlipkin.github.io/decoding/dec... And available for installation on PyPi: pypi.org/project/deco... 6/7

github.com

100

Researchers write arbitrary Python code and focus on expressing the core of their algorithms while deferring engineering details to the library With access to the right set of primitives, it becomes simple to express the ideas of many papers in a few dozen lines of code 5/7

100

Rather than prescribing a set of strict approaches, `decoding` supports a flexible framework for expressing computation that combines LLMs, external tools, symbolic verifiers, process reward models, LLM-as-judge, and anything else one can conjure up 4/7

100

At its simplest, you write a function that takes a string and returns a float, and we handle the rest If you'd like to step things up, we provide simple patterns to enable the efficient construction of powerful algorithms like Backtracking Monte Carlo Tree Search variants 3/7

110

Decoding is a library that makes it easy to solve difficult search and sampling problems Built around simple sample-and-rerank patterns that compose with arbitrary user-defined scoring functions and search strategies, a huge space of algorithms becomes easily expressible 2/7

110