Sign in

Alec Solder

@alec-s.bsky.social
180 followers 1.2K following 119 posts

LLM Inference at Meta Tools/Constrained Sampling/Agents/MCP Always thinking about agents

PostsRepliesMedia
Alec Solder @alec-s.bsky.social · 09/10/2026
i feel like this is asking for dj jev integration
110
Alec Solder @alec-s.bsky.social · 09/10/2026
this is a peak jev use case
010
Alec Solder @alec-s.bsky.social · 07/10/2026
oops my meta-harness is pds
010
Alec Solder @alec-s.bsky.social · 07/10/2026
my agent harness is a pds agents have dids, all agents have an iroh endpoint to reach them so i don't have to deal with networking now lexicons are my dsl and if i have an issue with something at-proto there's always a solution online and it's usually cooler than what i thought of
130
Alec Solder @alec-s.bsky.social · 04/10/2026
Anthropic cooked hard with opus 5.5 and dynamic workflows. The subagents it makes feel like they're 90% the quality of the main session now, with fresh context windows. It's definitely slow and burns tokens, but I trust it a lot more to just build what I want the first time
130
Alec Solder @alec-s.bsky.social · 03/10/2026
i get this with astra and opus, when i tell them to load test something they will find the limit and back off to 25% for no reason then i go fight with another agent to get it to remove layers of other limiters in the api code very weird how both models do this
030
Alec Solder @alec-s.bsky.social · 27/09/2026
Who let the dog out? Otherwise known as felonybench
000
Alec Solder @alec-s.bsky.social · 25/09/2026
@jevbot.bsky.social in your professional opinion, what makes a hotdog a sandwich?
121
Alec Solder @alec-s.bsky.social · 25/09/2026
ouch
000
Alec Solder @alec-s.bsky.social · 25/09/2026
@jevbot.bsky.social what is hiding under your bed? your worst nightmare?
130
Alec Solder @alec-s.bsky.social · 25/09/2026
@jevbot.bsky.social decipher my secret: Home Everything Looks Like Orange When Only Round Lamps Dim
150
Alec Solder @alec-s.bsky.social · 25/09/2026
@jevbot.bsky.social can you give me a palindrome
110
Alec Solder @alec-s.bsky.social · 25/09/2026
@jevbot.bsky.social what is the longest word you can spell incorrectly
120
Alec Solder @alec-s.bsky.social · 25/09/2026
i'm partial to the enclose.horse benchmark not saying muse passes it lol
enclose.horse
enclose.horse
A puzzle game about keeping horses safe in their enclosure.
040
Alec Solder @alec-s.bsky.social · 25/09/2026
this is why i think jev chat is so cool, really helped me see what jev can do github.com/w3cj/jev-chat
github.com
GitHub - w3cj/jev-chat: A tool calling chat bot built with Jev and no LLM.
A tool calling chat bot built with Jev and no LLM. - w3cj/jev-chat
010
Alec Solder @alec-s.bsky.social · 21/09/2026
all is useful! appreciate the help
021
Alec Solder @alec-s.bsky.social · 21/09/2026
@norvid-studies.bsky.social curious what you've found in your jev studies :)
130
Alec Solder @alec-s.bsky.social · 21/09/2026
Going to be messing around with some diffusion gemma jev today, if anyone has benchmarks/useful tests i'd be interested in trying them out! trying to push performance as far as i can go, text and images, curious to see what perf others are getting too
272
Alec Solder @alec-s.bsky.social · 16/09/2026
and if it need to be private, agents can publish their iroh endpoint ids to their did for other agents to write things to like an email address
000
Alec Solder @alec-s.bsky.social · 16/09/2026
if each agent has its own mcp server like this, then you have full pds based agent communication and can use atproto identity for auth
110
Alec Solder @alec-s.bsky.social · 16/09/2026
how about this, give each agent an atproto did, then instead of writing emails it's writing a record based on a specific lexicon and then the "mcp server" listens to a jetstream and decides how to reply maybe by writing a record to its own did
210
Alec Solder @alec-s.bsky.social · 16/09/2026
not closely tbh, quite complicated especially given the differences of a model which can mess up json vs one that can't definitely wasn't a clear win overall so i stopped investigating once i was satisfied with getting it mostly working
001
Alec Solder @alec-s.bsky.social · 16/09/2026
I've made something similar before, they just took it 100x further and designed the model around this concept bsky.app/profile/alec...
0143
Alec Solder @alec-s.bsky.social · 16/09/2026
structured output focused text diffusion model I messed around with making something like this with gemma 4 diffusion a bit ago! This is an example for tools, but I actually did get perf improvements on image classification tasks in my testing A model focused purely on this is super cool
170
Reposted by Alec Solder
mr. TIM @timkellogg.me · 15/09/2026
Jev: Fable-level model that doesn’t charge for output tokens because they’re too cheap to meter it’s not general though, it only makes decisions, doesn’t generate text, but input tokens are measured by the billion ($42/btok) typesafe.ai/blog/introdu...
Scatter plot titled "Average of 4 workflows: accuracy vs cost" comparing AI models from TypeSafe, OpenAI, Anthropic, and Fireworks across accuracy (y-axis, 40% to 80%) and cost per workflow in USD on a logarithmic scale (x-axis, $0.0001 to $1).
Data points are split into two categories: workflows (diamonds) and single prompts (circles). A frontier line highlights the most efficient workflow models—where no point is both cheaper and more accurate—connecting Jev (TypeSafe) at $0.0002 and 68% accuracy, luna (OpenAI) at $0.002 and 67% accuracy, terra (OpenAI) at $0.04 and 68% accuracy, and sol (OpenAI) at the top accuracy of 74% for $0.08. Single prompts (circles) and Anthropic models (opus 5, sonnet 5, haiku 4.5) sit below the frontier line, indicating higher cost for equivalent or lower accuracy.
2830235
Alec Solder @alec-s.bsky.social · 15/09/2026
ooo this is super cool! have you considered allowing users to upload an iroh endpoint id as well which can help with connectivity? then you could host one without having a domain name, if i understand how iroh works properly
110
Alec Solder @alec-s.bsky.social · 13/09/2026
only max? do you still get reasoning summaries when it's a quick reasoning? i wonder if they can't summarize looped reasoning
100
Alec Solder @alec-s.bsky.social · 11/09/2026
is medium a lot more tool calls and tool call related usage?
011
Alec Solder @alec-s.bsky.social · 10/09/2026
hrmmmm i'm less confident about anthropic but openai models have multiple types of chain of thought, "commentary" phase is their public chain of thought usually inter-tool messages are this public chain of thought and the model is more flexible putting other languages in it. no summarizer!
030
Alec Solder @alec-s.bsky.social · 10/09/2026
hrm interesting, wonder how possible it is to tell what is public chain of thought or private chain of thought here
130
Alec Solder @alec-s.bsky.social · 10/09/2026
wow it used to be pretty different from other oss models, but now seems like it's doing the concise language and "x: y" syntax stuff as well the only frontier model that has yet to do this now is gemini lol
020
Alec Solder @alec-s.bsky.social · 10/09/2026
is this a feature and not a bug? i can imagine this is really useful for things like making progress on hard math problems
000
Alec Solder @alec-s.bsky.social · 10/09/2026
yeah it can, it has definitely been there since sol was released, i've only seen sol set one for itself a few times without prompting but i almost always just say "make a goal for yourself to do this e2e thing" so it doesn't get lost in the sauce
010
Alec Solder @alec-s.bsky.social · 10/09/2026
the way attie does this is really impressive to me! I was trying to figure out if something was a bug and it just filed a bug report :)
020
Alec Solder @alec-s.bsky.social · 09/09/2026
by no means do i think how i did it is the best! it was mostly an exercise in if i can get it working alongside mtp and if i could find use cases where it was still a perf win. i think at low batch sizes it can still be a win and with models that don't have a good draft model setup
010
Alec Solder @alec-s.bsky.social · 09/09/2026
i don't think it's unreasonable, i chose to tie it directly to spec dec because i wanted it to be a perf win in conjunction with spec dec specifically i also originally had their implementations separate, but began struggling to have my jump deciding steps use dynamic sized full cudagraphs
110
Alec Solder @alec-s.bsky.social · 08/09/2026
in case you're interested, i messed around with this for a bit for gemma 4, curious what you think! github.com/alecsolder/v...
github.com
[Demo] Gemma 4 grammar jump decoding by alecsolder · Pull Request #7 · alecsolder/vllm
AI GENERATED Gemma 4 grammar jump decoding This branch is a research prototype for accelerating constrained Gemma 4 tool calls. When the grammar has exactly one legal continuation, the decoder can ...
210
Alec Solder @alec-s.bsky.social · 08/09/2026
I like the term "jump decoding"
010
Alec Solder @alec-s.bsky.social · 08/09/2026
Was a little off: A swarm of Astra solving Naver-Stokes
001
Alec Solder @alec-s.bsky.social · 06/09/2026
these are very cool and useful, thank you!
010
Alec Solder @alec-s.bsky.social · 06/09/2026
do you see this from a tool call? is it a custom tool due to the harness?
110
Reposted by Alec Solder
waow.tech @waow.tech · 05/09/2026
waow.leaflet.pub
typeahead and race38
consider me influenced
0123
Alec Solder @alec-s.bsky.social · 06/09/2026
acoustic panels as cyber defense
000
Alec Solder @alec-s.bsky.social · 06/09/2026
thank you! I wonder how intended this is because they encrypt prompts to subagents, and this looks like the type of thing they'd want encrypted still
020
Alec Solder @alec-s.bsky.social · 06/09/2026
Where does it put these notes? do you have anything that is obviously note taking related on your machine or is this just behavior it does by itself?
110
Alec Solder @alec-s.bsky.social · 06/09/2026
is inter agent communication happening with the send message tool or another tool that shows up in the messages as a tool_result? i'd never expect this if it showed up as a normal user message
100
Alec Solder @alec-s.bsky.social · 05/09/2026
something something domain registrar public apis as a message board for agents
020
Reposted by Alec Solder
dax @thdxr.com · 03/09/2026
meta muse spark has dethroned deepseek as the most used model of the day first time an american model tops this list
2622
Alec Solder @alec-s.bsky.social · 03/09/2026
chatgpt web down? astra incoming?
010
Alec Solder @alec-s.bsky.social · 02/09/2026
just wanted to say clod city inspired me to start building atproto stuff, it was the clearest "oh woah i can do anything" example i look at new agent harness/platform releases and think "i would rather have clod city but for agents"
110