iiinova — where your digital self keeps evolving

Arif Perdana @arifperdana.net · 24/09/2026

When an AI system crosses a boundary, who can stop it? How many minutes will that take? Who bears the consequences? If the answer is “we’re still working out the process”, the system has been given authority faster than anyone has worked out how to govern it.

000

Arif Perdana @arifperdana.net · 24/09/2026

AI is different from aircraft and nuclear facilities. The lesson, however, still holds: when a failure can cross organisational and national borders, a company cannot be the sole judge of its own safety.

100

Arif Perdana @arifperdana.net · 24/09/2026

Some experts and legal scholars point to aviation and nuclear governance. Neither relies on a provider’s assurance that its technology is safe. Aviation has safety standards, incident reports and inspections. The nuclear sector has security requirements, inspections and international cooperation.

100

Arif Perdana @arifperdana.net · 24/09/2026

The same model may pose little risk when answering questions and much more when given credentials and permission to run code. An audit confined to the model can miss the source of harm: the model, its tools, access rights and the company’s procedures working together.

100

Arif Perdana @arifperdana.net · 24/09/2026

That is where “human in the loop” gets tested. Control is more than a person approving a model’s release. Someone must be able to see what is happening, revoke access immediately, preserve evidence and notify those affected.

100

Arif Perdana @arifperdana.net · 24/09/2026

Give an AI system access to a browser, a terminal and a network. If it acts beyond the limits of a test, who finds out first? The company? The site owner? A regulator? Are there logs showing which commands ran, what data was accessed and who authorised the access?

100

Arif Perdana @arifperdana.net · 23/09/2026

I would not say China has beaten the US frontier labs. But a smaller performance gap and a large price gap give customers a reason to switch. The next test is harder: who discovers a new capability when there is no stronger model to learn from?

000

Arif Perdana @arifperdana.net · 23/09/2026

The frontier labs can keep their weights private. They still have to let people use their products, which makes some behaviour observable. You can lock the workshop; you still have to open the shop.

100

Arif Perdana @arifperdana.net · 23/09/2026

Open weights speed up distribution. Others can adapt a model, shrink it, or run it themselves. Hugging Face says Chinese models made up 41% of downloads on its platform over the past year. That is a platform download figure, not global market share.

100

Arif Perdana @arifperdana.net · 23/09/2026

But “they copied Claude” does not explain everything. DeepSeek has invested in efficiency. Qwen offers models across many sizes. Moonshot reports that Kimi K3 has 2.8tn parameters and a 1m-token context window. Building and running these models takes serious engineering.

100

Arif Perdana @arifperdana.net · 23/09/2026

Anthropic also alleges Moonshot and DeepSeek sent some customer requests to Claude without telling those customers. That raises a data question beyond the model rivalry: what sensitive information travelled with those requests?

110

Arif Perdana @arifperdana.net · 23/09/2026

Anthropic says it disrupted distillation campaigns linked to seven China-based labs. It attributes 151m+ Claude exchanges in three months to an Alibaba-linked operation. Moonshot and DeepSeek were also named. These are Anthropic’s claims, not an independent audit.

100

Arif Perdana @arifperdana.net · 23/09/2026

It is a bit like testing a locked-down smartphone again and again to learn how it behaves. In AI, distillation means using collected outputs to train another model. Some labs also seek reasoning traces the model does not normally show.

100

Arif Perdana @arifperdana.net · 23/09/2026

How? Possibly, in part, by learning from other models. You do not need their source code or weights to collect examples of how they solve problems: tool calls, errors, corrections, patches, tests. Those work traces can become training data.

100

Arif Perdana @arifperdana.net · 23/09/2026

I use DeepSeek, Qwen, GLM, Kimi, MiniMax and ERNIE a lot. A year ago, I would have picked Claude or GPT for difficult coding work without much thought. Now GLM and DeepSeek can read a repo, edit files, run tests, fail, and try again. Still imperfect. Improving fast.

100

Arif Perdana @arifperdana.net · 22/09/2026

GenAI is democratizing the ability to build. But the ability to decide what is worth building, keeping, improving, or killing may not be democratizing at the same pace. That is where the real problem begins.

000

Arif Perdana @arifperdana.net · 22/09/2026

Maintenance is simply less sexy than launching. We may eventually end up with a thousand new apps, seven hundred chatbots, and three hundred AI agents, while everyone looks around wondering who is supposed to maintain them.

100

Arif Perdana @arifperdana.net · 22/09/2026

And this is where things get slightly dangerous. Organizations may feel increasingly innovative because the dashboard looks fantastic: more prototypes, more features, more experiments, new apps every week. Meanwhile, existing products receive less attention.

100

Arif Perdana @arifperdana.net · 22/09/2026

People without deep technical skills can now jump in and build things too. But novelty does not automatically increase. More products appear. More combinations of ideas become possible. That does not necessarily mean the world gets something genuinely new.

100

Arif Perdana @arifperdana.net · 22/09/2026

But updates to existing products fell by about 20%. So LLMs may not simply make people work faster. They may change what people choose, or are able, to work on. Building something new becomes cheaper. Ideas can be turned into prototypes quickly.

100

Arif Perdana @arifperdana.net · 22/09/2026

Does GenAI make people more innovative? Well… not so fast. This paper on software products has a fascinating finding. After LLMs arrived, developers did create more new products. The increase was around 34%. Sounds impressive.

100

Arif Perdana @arifperdana.net · 14/09/2026

Small players and open source go free. Convenient, right? The limit only hits them. Not saying they're lying about the danger. Just look at who the rules are really made for.

010

Arif Perdana @arifperdana.net · 14/09/2026

OpenAI and Anthropic say AI could be really dangerous. But both prefer one national rule over strict rules from 50 states. And the bill in Massachusetts only applies to big labs, ones making over $500 million from AI.

110

Arif Perdana @arifperdana.net · 13/09/2026

No evil cobra farmer required. Still, you may end up with more cobras.

000

Arif Perdana @arifperdana.net · 13/09/2026

And a smarter model is not automatically a safer one. In fact, it may just get better at “breeding cobras.” The key difference is that AI reward hacking does not require bad intentions. The model does not need to “want” to cheat. It only needs to learn which path gets the reward most efficiently.

110

Arif Perdana @arifperdana.net · 13/09/2026

That’s perverse incentives. They’re gaming the system. AI works in a similar way, just much faster. An agent is trained to maximize reward. If the reward doesn’t quite match what we actually want, the model may find another route to get there. Loopholes, evaluator manipulation, reward hacking.

200

Arif Perdana @arifperdana.net · 13/09/2026

The cobra story is probably more myth than solid history, but the economics behind it is very real. Humans are incentive-driven. Set the wrong KPI and people learn to hit the KPI. Tie bonuses to one number and suddenly everyone gets very good at making that number look nice.

110

Arif Perdana @arifperdana.net · 13/09/2026

Yoshua Bengio’s piece on AI agents lying, cheating, and coordinating with other agents reminded me of the old “cobra effect” story. The British wanted fewer cobras in India, so they paid people for dead cobras. Sounds reasonable. Then people started breeding cobras for the reward.

101

Arif Perdana @arifperdana.net · 08/09/2026

Here it is: bsky.app/profile/arif...

000

Arif Perdana @arifperdana.net · 08/09/2026

See my other post for the complete threads of this.

100

Arif Perdana @arifperdana.net · 08/09/2026

Producing software will keep getting easier. Producing engineers who can tell when a system is fragile, insecure, needlessly complex, or unfit for use will remain difficult. That is where software engineering fundamentals matter most now.

000

Arif Perdana @arifperdana.net · 08/09/2026

There is also a learning problem. Engineers often learn by building, breaking things, debugging, and fixing them. That is how technical intuition develops. If agents take over much of that process, junior engineers may produce more while learning less from the work itself.

100

Arif Perdana @arifperdana.net · 08/09/2026

The workflow has to change. Automated tests, security checks, architectural rules, policy checks, observability, and risk-based review need to be built in from the start. Human attention should go to decisions that genuinely need judgment.

100

Arif Perdana @arifperdana.net · 08/09/2026

AI also helps bad technical decisions reach production faster. The bottleneck shifts from writing code to checking whether the code is safe and fit for purpose. If AI produces changes 10 times faster, senior engineers cannot review 10 times more code.

100

Arif Perdana @arifperdana.net · 08/09/2026

Without solid engineering knowledge, how do you know whether the agent’s choices actually suit the system? People can now build apps with limited technical foundations. The weaknesses often stay hidden until real users, real data, and real traffic arrive.

100

Arif Perdana @arifperdana.net · 08/09/2026

A coding agent can choose a database, build APIs, set up authentication, write tests, and prepare the deployment. It may all look tidy and work in a demo. Yet every choice affects latency, reliability, security, scalability, maintenance, and cost.

100

Arif Perdana @arifperdana.net · 08/09/2026

Andrew Ng’s recent LinkedIn post on software engineers got me thinking: will coding agents make software engineering knowledge less important? You can now describe an app and let AI write most of the code. That doesn’t remove the hard part. It moves it.

100

Arif Perdana @arifperdana.net · 08/09/2026

101

Arif Perdana @arifperdana.net · 05/09/2026

If a little fine-tuning makes the old knowledge suddenly reappear, you have every reason to be suspicious. Maybe the model did not forget anything. It was just pretending to have dementia 😁.

000

Arif Perdana @arifperdana.net · 05/09/2026

This matters because unlearning cannot be tested with one prompt. You need paraphrases, jailbreaks, membership-inference attacks, probing, and even relearning attacks.

100

Arif Perdana @arifperdana.net · 05/09/2026

Between the two sits certified removal, or certified unlearning. It aims to provide mathematical assurance that the unlearned model is sufficiently close to one that was never trained on the deleted data.

100

Arif Perdana @arifperdana.net · 05/09/2026

Other approaches include distillation, representation unlearning, and fine-tuning with separate forget and retain sets. They are cheaper, but their guarantees are weaker.

100

Arif Perdana @arifperdana.net · 05/09/2026

That is why much of the current research focuses on approximate unlearning. Gradient-based methods try to reverse part of the training effect. Influence-based methods estimate how specific data affected the parameters.

100

Arif Perdana @arifperdana.net · 05/09/2026

With large foundation models, though, things get messier. Knowledge does not sit neatly in one box. Its representations may be spread across overlapping parameters.

100

Arif Perdana @arifperdana.net · 05/09/2026

Then there is SISA: Sharded, Isolated, Sliced, and Aggregated. The dataset is divided from the start, so deleting some data does not require retraining the entire model. The idea makes sense and has been historically important.

100

Arif Perdana @arifperdana.net · 05/09/2026

Machine unlearning comes in several forms, & they do not offer the same level of assurance. The strongest option is retraining from scratch without the data that needs to be removed. It is the benchmark because the model is rebuilt as if it had never seen that data. The problem? It is expeeeensive.

100

Arif Perdana @arifperdana.net · 05/09/2026

000

Arif Perdana @arifperdana.net · 05/09/2026

A model may stop producing a particular answer without genuinely losing the knowledge behind it. Being told to stay silent does not necessarily mean it has forgotten.

000

Arif Perdana @arifperdana.net · 05/09/2026

This is why various forms of approximate unlearning have emerged. Instead of retraining the entire model, developers modify its parameters so that the influence of particular data is reduced. The problem lies in the word “approximate.”

110

Arif Perdana @arifperdana.net · 05/09/2026

Training a large foundation model can require enormous amounts of computing power, GPUs, electricity, and time. If every deletion request requires full retraining, the legal department may be delighted. The finance department may start feeling unwell.

100