Throx comments

isabelleToday  07:20 pm JST

Distillation is forbidden by OpenAI's terms of service. If DeepSeek did that, it is violating those terms.

I am not arguing that this was legal or illegal, I am not educated enough in law to do this (although knowing about the debate of OpenAI not following other companies terms either makes me wonder... but that's a different topic). My point was purely technical speaking, it's an achievement which OpenAI themselves were not able to achieve (but now presumably can, because DeepSeek published paper on how)

So again, I'm curious what the investigations and legal discussions will show.

TaiwanIsNotChinaToday  04:57 pm JST

My understanding is that DeepSeek lifted the model, the part that is generated after the training. How they got access to it, I don't know.

I assume you are referring to the knowledge distillation. Simply saying, that means a larger network (teacher) teaches a smaller network (student) to do same things. It's quite common actually. So common, that even OpenAI probably does that similar as Meta. Let's assume the worst case that teacher was OpenAI's model and no other source, then still the technical capability to train a much smaller student is a breakthrough done by small startup. Because OpenAI did not achieve that (their model is larger and requires more compute power = money) and I am pretty sure they are reading the paper that DeepSeek published. In that hypothetical case, the main training (for the teacher) could be skipped and the managed with restricted GPU access to train their smaller model and the legal question of using OpenAI output becomes question, but technically still wow

I think there are several overlapping items:

OpenAI claims they have used ChatGPT output to train. Even if that is true, ChatGPT output is basically posted by thousands / millions of humans who use it. So that either way will happen someday.

Then again, OpenAI also just used everything it found on the internet, ignoring any legal responsibility.. like the article mentioned.

But more technically speaking, the interesting part of DeepSeek is that they can do (almost?) same as OpenAi's model with less compute. That is an achievement, because clearly OpenAI could not or they would just have wasted money with compute power. So regardless of what data has been used and what country that company/startup is from, just the fact that a startup can beat them here is astonishing

semperfiMay 1  07:39 am JST

Ummm - such a hypocritical double standard !

What about China !????

Though NOT a G7 country - but hey - WHERE is the International censor on China's massive use of coal for energy ?

China is the world's largest producer and consumer of coals .

Her use has almost QUADUPLED since 1990 - so to date 56% of China's energy is Coal

As the another comment was saying, finger-pointing itself is already not a good way to argue anything.

But another aspect is that China reduced their coal consumption in last 10 years and replaced (mostly) with renewable energy. (Source: China: Energy Country Profile - Our World in Data)

Simply said, it's not as easy as you make it seem