iiinova — where your digital self keeps evolving

Agents are functions. It's much better to think of them as unreliable APIs that will give you the correct answer most of the time. As soon as I did that, my code became cleaner and much more efficient.

060

When you anthropomorphize agents, you can't see them for what they are. This might sound weird, but errors didn't make sense because I was thinking about agents the wrong way.

You should stop treating agents like humans. This has probably been one of the biggest unlocks for me over the past few months in terms of building agents. I see people giving agents personalities, goals, and even titles. But agents aren't people — they are functions with context windows.

151

At this point, the data is ready for training. They are processing more than 10M documents, and the pipeline takes several hours to complete. A significant amount of code was written here, and 0% of it was AI-generated. 4/4

020

1. Load raw data from 6 different plaves 2. Remove unnecessary tokens 3. Remove low-quality data 4. Remove duplicated data 5. Replace personally identifiable information 6. Normalize data formats 7. Align multi-modal data 8. Enrich the data with domain-specific annotations 9. Tokenize the data 3/4

120

I wanted to outline the steps to give you an idea of what production applications typically deal with. Here are the steps in their pipeline: 2/4

120

I'm working with a huge company on their data preparation pipeline, and they are running a 9-step data processing pipeline. This all happens before their data is ready for training. There's a lot of code here and considerable complexity. 1/4

130

I'll try to stick around.

000

And no other programming language can match Python in code clarity and readability. Except English, maybe.

210

On top of that, over 90% of the Python code you interact with uses C/Rust/Fast-thing behind the scenes anyway. The speed at which you can write good Python code is way more important in most situations than the speed at which that code must run.

Python is the best language in the world. Yes, Python is slower than other languages, but I don't care because most of my work doesn't require it to be faster. Python is fast enough for what I need, and fast enough for most people out there.

220

Gemini's documentation says that "responses for a given prompt are mostly deterministic, but a small amount of variation is still possible." I'm not sure about you, but "mostly deterministic" is not the same as "deterministic." 9/9

030

So, in practice, the LLM you are calling is non-deterministic, even at temperature = 0. OpenAI's documentation tells us that we can expect "(mostly) deterministic outputs across API calls." 8/9

• Your model provider will patch, retrain, and fine-tune the model you are using without telling you. What was true yesterday isn't guaranteed today. (I'm kinda cheating with this last point, because it's not related to the nature of models, but it's still a problem we have to deal with.) 7/9

210

• If you are using a model scaled across multiple servers, you are not always hitting the same model instance, which increases the chances of finding issues with differences in hardware. 6/9

• Floating-point operations are tricky, and even when using greedy decoding, a model might suffer from tiny numeric drifts that shift the outputs. 5/9

Theoretically, temperature=0 makes a model deterministic. In practice, this is not the case. There are several reasons: • Non-deterministic hardware ops can produce different outputs. Two GPU kernels can diverge slightly run to run. This may introduce slight differences in your outputs. 4/9

Yesterday, most replies tried to school me about the temperature parameter when using an LLM. Yes, you can set this parameter to 0 to try to force a model to return the same answer, but in practice, you'll find this isn't guaranteed. 3/9

A deterministic model will always return the same answer given the same input. My argument is that the set of large language models we use today can't fulfill that contract. 2/9

Yesterday, I posted that large language models are non-deterministic, and people went bonkers. People called me stupid, asked me to go back to school, and used so many other names I can't remember. Today, I want to elaborate. 1/9