iiinova — where your digital self keeps evolving

Scott Condron @scottcondron.bsky.social · 09/03/2025

How do I get Bluesky to show me less politics and more AI/ML things? I have followed mostly people who work in AI/ML

Scott Condron @scottcondron.bsky.social · 09/03/2025

Maybe they could tell you what they’ve learned like “it seems you’re interested in staying up to date with recommender systems, want to add that to your feed?”

000 0

Scott Condron @scottcondron.bsky.social · 29/11/2024

Prompts within a complex system are brittle I have seen some teams be successful by replacing prompts with smaller, more deterministic components and improved reliability with fine-tuning. Anyone else have success with this approach? Seems to help a lot with agents

010 0

Reposted by Scott Condron

Cathy Wu @cathywu.bsky.social · 27/11/2024

I collected some folk knowledge for RL and stuck them in my lecture slides a couple weeks back: web.mit.edu/6.7920/www/l... See Appendix B... sorry, I know, appendix of a lecture slide deck is not the best for discovery. Suggestions very welcome.

web.mit.edu

311418 0

Scott Condron @scottcondron.bsky.social · 28/11/2024

If you’re taking time to enjoy your family and not building with LLMs, you’re ngmi. America is cooked

020 0

Scott Condron @scottcondron.bsky.social · 26/11/2024

LLM app dev broke our comparison tools because tiny diffs can cause large behaviour change. At wandb, we've spent years thinking about experiment comparison. We've added new tools for LLM app dev: code, prompts, models, configs, outputs, eval metrics, eval predictions, eval scores.. wandb.me/weave

180 0

Scott Condron @scottcondron.bsky.social · 25/11/2024

The art of how to refer to model behaviour with tasteful non-person metaphors. Say “stochastic” you’re in one camp, say “emergent” you’re in another. It’s a minefield out there people

000 0

Scott Condron @scottcondron.bsky.social · 23/11/2024

People ask for an iOS app but maybe we shouldn’t as it would cause more misery on-the-go

000 0

Reposted by Scott Condron

Eugene Vinitsky 🍒 @eugenevinitsky.bsky.social · 20/11/2024

Being logged into wandb on your phone is a recipe for misery

9754 0

Scott Condron @scottcondron.bsky.social · 23/11/2024

Would be happy to schedule a chat to hear more about your experience with W&B

000 0

Scott Condron @scottcondron.bsky.social · 23/11/2024

hey, sorry to hear your complaints about wandb. Have you seen the big response in that issue with options? Tables is built on parquet so it’s difficult from an architectural perspective. With the recent release Weave, there may be a path forward by using the weave backend instead of parquet…

100 0

Scott Condron @scottcondron.bsky.social · 23/11/2024

Agreed, fellow competitor. It’s the biggest hurdle I see from teams trying to build GenAI features We need tools to lower the barrier to entry with LLM judges, existing benchmarks, manual annotation as eval collection, synthetic data… anything else?

010 0

Scott Condron @scottcondron.bsky.social · 22/11/2024

I think these small models are not for day to day use but instead, they’re for b2c applications of LLMs, where it’s cost/latency prohibitive to use anything else

000 0

Scott Condron @scottcondron.bsky.social · 21/11/2024

- it's much better for scraping if the links included are .md files - you need to be clear which files to include and which are optional because context blows up quickly - automating creating your docs' llms.txt is pretty easy

130 0

Scott Condron @scottcondron.bsky.social · 21/11/2024

Lessons from creating an llms.txt file An llms.txt file is a way to tell a LLM about your website. In the .txt file, you include links to other files with info to learn more. - the llms.txt file isn't the file you send to an LLM, you use it to generate a llms .md file

130 0

Scott Condron @scottcondron.bsky.social · 20/11/2024

from @hamel.bsky.social’s hamel.dev/blog/posts/llm… We're building LLM / Human "scorers" in @weightsbiases.bsky.social to have the same data model for this reason

hamel.dev

120 0

Scott Condron @scottcondron.bsky.social · 20/11/2024

Your human and LLM judges should follow the same criteria. Then, you can transition from manual to automated evaluation once you have inter-annotator agreement between LLM & human. You now have a faster iteration speed and the annotator can focus on finding edge cases!

140 0

Scott Condron @scottcondron.bsky.social · 19/11/2024

The most bizarre AI interview I've ever done was at wandb when as usual I asked a candidate to build an AI classifier in any language/framework of their choice.. And they nonchalantly said "I'll write it in Redstone", to which I almost let loose a chuckle until...

020 0

Scott Condron @scottcondron.bsky.social · 19/11/2024

Claude defaults to concise responses when there's high demand, clever way to smooth peaks

050 0

Reposted by Scott Condron

Alex Volkov (Thursd/AI) @altryne.bsky.social · 18/11/2024

We've been working on just that at @weightsbiases.bsky.social with Weave! Weave is a lightweight llm tracing and evaluations toolkit, that focuses on letting you iterate fast and make sure that your production LLM based application is not degrading when you change prompts or models!

4143 0