iiinova — where your digital self keeps evolving

okay let's see what bugs come up lol 🤞

Screenshot from google analytics showing 60 active users per minute

Finally, if we need help with a term or character that was previously mentioned, Dewey can help with a summary of the term so we don’t have to look it up ourselves.

If you've stopped reading a book for a while, it can be challenging to pick it up again and remember what you've read. To help with this, it can help with summarizing the book up to the current page and refresh our memory, highlighting major themes, characters, and concepts.

It can also help with creating quizzes / flashcards. The goal here is to test our knowledge and improve retention.

With the context, it can answer simple queries via "Explain" and "Discuss". The goal is to keep us in flow while reading, instead of having to reread other sections of the book or open a web browser for our queries.

Eugene Yan @eugeneyan.com · 19/12/2024

At the heart of AI Reading Club is Dewey, your AI reading companion. It understands context via selected text or the page we're on. This explicit context is displayed during discussions. At the same time, behind the scenes, it can retrieve and consider the rest of the book as implicit context.

It understands our context either via the text we select or the page we're on. This explicit context is displayed during discussions. At the same time, behind the scenes, it can also retrieve and consider the rest of the book as implicit context.

130

phone usage has halved since starting december detox and deleting all social media apps off my phone

2210

to the latter point, the anti-ai comment were really strong in several comments, even those that didn't have anything to do with ai (these images are part of the appendix of the writeup)

comments that talk about the inevitability of ai failing

also, while _some_ accounts that didn't like their data being scrapped had anti-AI explicitly posted on their profiles, not all of did. i hope i expressed this nuance sufficiently, and not a sweeping "people objecting to their data being stolen without permission as anti-AI"

To validate this, I visited a sample of the critique accounts and found at least a dozen profiles of writers, artists, musicians, and other creatives, some of whom added explicit anti-AI declarations to their bios.

100

ah i see your point now, thank you for clarifying! my point was that there was no stealing of data from bluesky's database, and no scraping of html. instead, the data was simply downloaded via the api. i deliberately avoiding comment on license or legality.

At first glance, the primary issue seemed to be the lack of consent in data collection. Some comments expressed anger at having their data “scraped” or “stolen” without permission. (Note: The accusation is incorrect—there was no scraping or stealing involved. Posts on Bluesky are public and the Bluesky firehose API is open for all to consume.)

000

Eugene Yan @eugeneyan.com · 09/12/2024

oh, perhaps it's just me that isn't used to such reactions to the release of a dataset, and the comments against the training of AI. haven't come across such negative reactions elsewhere tbh

Reactions against the release of one-million-bluesky-posts

responses objecting to the dataset being used as training data

120

Day 1 of hitting the slopes this winter 🏂

1140

Eugene Yan @eugeneyan.com · 07/12/2024

Repeat after me: I will build evals for my tasks. I will build evals for my tasks. I will build evals for my tasks.

4649

Eugene Yan @eugeneyan.com · 07/12/2024

Learning about quantization suffixes while `ollama pull llama3.3` download completes (fyi, quantization for the default 70b is q4_K_M) • make-ggml .py: github.com/ggerganov/ll... • pull request: github.com/ggerganov/ll...

Old quant types (some base model types require these):
- Q4_0: small, very high quality loss - legacy, prefer using Q3_K_M
- Q4_1: small, substantial quality loss - legacy, prefer using Q3_K_L
- Q5_0: medium, balanced quality - legacy, prefer using Q4_K_M
- Q5_1: medium, low quality loss - legacy, prefer using Q5_K_M

New quant types (recommended):
- Q2_K: smallest, extreme quality loss - not recommended
- Q3_K: alias for Q3_K_M
- Q3_K_S: very small, very high quality loss
- Q3_K_M: very small, very high quality loss
- Q3_K_L: small, substantial quality loss
- Q4_K: alias for Q4_K_M
- Q4_K_S: small, significant quality loss
- Q4_K_M: medium, balanced quality - recommended
- Q5_K: alias for Q5_K_M
- Q5_K_S: large, low quality loss - recommended
- Q5_K_M: large, very low quality loss - recommended
- Q6_K: very large, extremely low quality loss
- Q8_0: very large, extremely low quality loss - not recommended
- F16: extremely large, virtually no quality loss - not recommended
- F32: absolutely huge, lossless - not recommended

3234

Eugene Yan @eugeneyan.com · 03/12/2024

Here's my attempt at something similar—machine learning systems and applications in industry—a couple years ago. Isn't as fancy as the one above though lol applyingml.com/papers/

2130

Eugene Yan @eugeneyan.com · 03/12/2024

Wow, this is such a useful resource of industry LLM applications! And filtering via search/tags is so responsive. I was thinking of compiling something like this over the holidays (ala applied-ml) but thanks to @strickvl.bsky.social I can spend the time reading instead ♥️ zenml.io/llmops-datab...

UI for the LLMOps Database with the search query of "eval"

3504

Eugene Yan @eugeneyan.com · 01/12/2024

@hamel.bsky.social is 💯: hamel.dev/blog/posts/a... Write for yourself. Assume no one reads it. Write about topics to learn, clarify your thoughts, put out a bat signal. This makes it sustainable. Writing is primarily a single player game; multiplayer benefits (e.g., audience building) are bonuses.

The key is authenticity. Don’t do this just for marketing—do it because you’re genuinely interested in learning from others and building on their ideas. It’s not hard to find things to be excited about. I’m amazed by how few people take this approach. It’s both effective and fun.

4595

Eugene Yan @eugeneyan.com · 30/11/2024

Great explainer on sinusoidal positional encoding and rotary positional embedding (RoPE). fleetwood.dev/posts/you-co...

2616

Eugene Yan @eugeneyan.com · 28/11/2024

This Thanksgiving, I'm grateful for peace, health, and happiness for my family and me—what are you thankful for?

Nassim Taleb on 'True Wealth'
Vishal Khandelwal, safalniveshak.com
Nassim Taleb is one of my favourite authors, and his Antifragile is one of my favourite books. One of this book's chapters that interests me particularly is titled 'Via Negativa'. Here, Taleb argues that the solution to many problems in life is by removing things, not adding things.
For example, here is a list of things Taleb counts as constituents of true wealth that are all about subtracting things (via negativa) from life than adding -

Worriless sleeping
Clear conscience
Reciprocal gratitude
Absence of envy
Good appetite
Muscle strength
Physical energy
Frequent laughs
No meals alone
No gym classes
Some physical labor
Good bowel movements
No meeting rooms
Periodic surprises

I could check twelve from this list (let the ones I didn't check remain a secret).
What about you? What in the list remains getting checked for you?

11413

Eugene Yan @eugeneyan.com · 27/11/2024

And if you're looking for more learning over the long thanksgiving weekend, this could be a good place to start: eugeneyan.com/start-here/

Writeups on machine learning systems, techniques, ml & engineering, and weekend prototypes

1145

Eugene Yan @eugeneyan.com · 27/11/2024

Feels good to be mentioned on HN for engineers learning AI 🥰 Helping others is a big reason I write. Here's a list on ML/AI: ## Building AI systems • Patterns for Building LLM-based Systems: eugeneyan.com/writing/llm-... • What We’ve Learned From A Year of Building with LLMs: applied-llms.org

Read through this making flashcards as you to: https://eugeneyan.com/writing/llm-patterns/
Then spin up a RAG-enhanced chatbot using pgvector on your favourite subject, and keep improving it when you learn about cool techniques

---

Lots of people can get impressive demos up and running, but if you want to run AI products in production, you're going to have to do system evals. System evals make sure your product is doing what it says on the box with unquantifiable qualities.
We wrote a zine on system evals without jargon: https://forestfriends.tech
Eugene Yan has written extensively on it https://eugeneyan.com/writing/evals/
Hamel has as well. https://hamel.dev/blog/posts/evals/

3709

Eugene Yan @eugeneyan.com · 27/11/2024

yay Latte welcomes you!

020

Eugene Yan @eugeneyan.com · 23/11/2024

It's helpful to distinguish between crisp vs. fuzzy tasks: • Crisp: Answers are verifiable, like math or code • Fuzzy: Answers are subjective, like reasoning, summarization, translation, multi-turn dialogue The latter is far harder to evaluate reliably aligned.substack.com/p/crisp-and-...

This is a taxonomy for task space that I find useful when thinking about what we need to do to solve alignment.

Crisp tasks are more reasoning/system-2 based. Whether a response is good is typically precisely defined. Reasonable, knowledgeable people don’t disagree about it.
Examples: math, coding competitions, physics questions, logic puzzles, basic factual knowledge, …
Fuzzy tasks are more intuition/system-1 based. Whether a response is good is typically somewhat vaguely defined or could fall within a range. Reasonable, knowledgeable people may disagree about it.
Examples: distinguishing cats and dogs in pictures, recognizing strong go moves, poetry writing, assistant helpfulness ratings, …
In contrast to fuzzy logic, where this name is borrowed from, here crisp and fuzzy don’t refer to the output space: preferences comparisons have a small number of discrete allowed values, and image classification typically uses a discrete space. Crisp tasks like coding competitions have a output space that involves hundreds of discrete tokens, and this is the same output space of poetry writing, a fuzzy task.

There are many crisp tasks that we can evaluate very reliably, and thus it’s feasible to train on them extensively. In contrast, fuzzy tasks are often particularly difficult to evaluate reliably, and we currently don’t know how to do this at a superhuman level.

3470

Eugene Yan @eugeneyan.com · 22/11/2024

Hey Bluesky, meet swirly-green-with-a-touch-of-pink sky

Northern lights, streak of green with an igloo

Northern lights, swirls of green over igloos

Northern lights with streaks of green and a touch of pink

Northern lights, streak of green and some pink

2481

Eugene Yan @eugeneyan.com · 20/11/2024

yea some standard commands / packages need workarounds for fish, such as rbenv

brew install chruby-fish ruby-install ruby-build
brew install rbenv

# workaround for fish shell
set --universal fish_user_paths $fish_user_paths ~/.rbenv/shims
rbenv global 3.3.5
rbenv rehash

jealous! here’s my view from the plane lol

probably data, evals, and the flywheel that mixes the cake batter

Potentially nitpicky but competitive advantage in AI goes not so much to those with data but those with a data engine: iterated data aquisition, re-training, evaluation, deployment, telemetry. And whoever can spin it fastest. Slide from Tesla to ~illustrate but concept is general

Also see Karparthy’s take on it

It’s hard to understand now, the Atari RL paper of 2013 and its extensions was the by far dominant meme. One single general learning algorithm discovered an optimal strategy to Breakout and so many other games. You just had to improve and scale it enough. My recollection of the memetics is that Yann LeCun was one prominent person who really didn’t care much and talked about the cake over and over again, where RL was just the final cherry on top with representation learning as the meat and supervised learning the icing, and he was conceptually exactly right about that at least with today’s stack and hindsight (pretraining = meat, SFT = icing, RLHF = cherry, ie the basic ChatGPT training pipeline). Which is fun because today he really doesn’t care much for LLMs either. (But for reasons that I tbh don’t always fully follow.)

2230