Open Weight models : Who should decide what we can use

Fear. Fear is normally an unpleasant emotion that arises when we project or “know” danger is around the corner. Fear is a great emotion in that if a tiger is about to pounce it gives us an adrenaline rush to get the F outa there. The problem with fear is that people will use possibilities of danger to politically move people from one side to the other of a certain topic. 

You may be thinking to yourself, “I thought this was a blog on AI Governance”...Well today we are talking about Open weight models and who if anybody is using fear to tell you what you can and can’t use.  

The Open Weight model conversation Blew up as a Chinese open weight Model Kimi K 3 seemingly performs at frontier level models (anthropic and open ai is the current frontier). The weights for the training are promised in a few days and it became a powder keg. First, open weight is essentially open source. An open weight company shares how it trained its AI to get the results it has. Open AI and Anthropic, United States (and world) current frontier models are not open weights.

We have this Open Weight company, that is Chinese, that has joined the head of the pack. Then you have an open AI’s head of strategic futures Dean Ball (cool job title), who I guess has only been on the job a few weeks, who hops on twitter and says this: He is personally surprised the Chinese state allows open-sourcing models this good, attributing it partly to the CCP lacking "AGI-pilled" views and partly to China's compute shortage making open weights an unintended byproduct of US export controls. He then went on to infer that open source is “inherently decelerationist” (which is kinda ridiculous to say, but I digress)

There is a lot to unpack here. First, this is a Chinese which takes the fear narrative up 50% off the bat. Next, to me is that AGI Pilled comment. To me it reads that everybody should base their AI strategy on the fear of AGI instead of the excitement of a new tool. The unintended consequence of US policy on Chinese chips is an interesting take and maybe that has something to do with it…that is outside my scope here. 

To be clear, fear of open weight models are a list of doomsday scenarios. Examples: AGI with no shut off switch that a 12 year old in china creates. Or everyone in the world has access to how to build a dirty bomb. Or everyone is a top level hacker. The list goes on but you see my point.

Scary stuff certainly. Also scary is keeping that power in the hands of a few people and thinking things like this can never leak. At the heart of it, open weight models are irreversible to the public hands with safety training that is reversible. Fear runs high.

Let’s get into the positives of open weight models. In fact, I will use fear from this lens as well. If all our companies are closed models, you put enormous power in the hands of just a few companies. AGI in just Sam Altmans hands…that is scary. You can’t audit what you can’t inspect. Transparency keeps big brother at bay. If everything can be done in the shadows the level of shadowing things done increases. To this point, security through the scrutiny of 1 million eyeballs looking at something. Of course companies have security teams of their own looking at risk but this pales in comparison to having those security teams AND having the nerds in the public comb through everything.

Open source anything grows with more creativity. Uses aren’t just around what makes the most money or soon share holders the most money. People build for what challenges they are facing.

Last but not least, Sovereignty and resilience. Countries and enterprises don't want critical infrastructure dependent on a US company's API that can be revoked, repriced, or geofenced. Open weights run on your own hardware and offline if you choose. Now, from a wisdom perspective what can or should be done. Both sides make great points. The AI steward answer: Fear should have a seat in the car but never be the one driving. Capability diffuses regardless. Deepseek and Kimi K 3 showed frontier-adjacent models can be trained relatively cheaply. If openness is inevitable, better to have it happen with Western norms, documentation, and safety tooling attached. 

These types of governance questions are why I find the topic so fascinating. There is no “right” answer. However, I trust the answer that has been vetted by many vs the answer that was created by a guy, in an office, appointed by a guy who got the job in a popularity contest.