Tag Archives: token

OpenAI’s latest blunder shows the challenges facing Chinese AI models

OpenAI’s latest blunder shows the challenges facing Chinese AI models

In fact, among the few long Chinese tokens in GPT-4o that aren’t either pornography or gambling nonsense, two are “socialism with Chinese characteristics” and “People’s Republic of China.” The presence of these phrases suggests that a significant part of the training data actually is from Chinese state media writings, where formal, long expressions are extremely […]