Trusting China at the End of the World (Part Three)

In Part 1 I introduced artificial intelligence research and its dangers. Perhaps artificial super-intelligence will come about so quickly that, unless we are extremely and immediately careful, it destroys us. But perhaps, as some experts predict, it will come about slowly and gradually. Maybe we’ll see its rise coming, and have a few decades to get ready by coordinating with China to ensure adequate safeguards.

In Part 2 I looked at the differences between AI and human diplomacy. We have considerable evidence that artificial intelligence is able to make binding promises and be truthful and accurate about its intentions in ways it can make clear to potential deal-partners. As such, it has a massive advantage over humans, who have a great deal of trouble convincing others that they are trustworthy. From the vantage point of 2025, it seems a confederation of artificial intelligences is a far more likely prospect than the sort of Sino-American relationship necessary to prevent artificial superintelligence from arising in the first place.

China is investing heavily in AI, with impressive but somewhat chaotic results. Despite American efforts, Chinese advances in military AI are clipping along rapidly. Chinese drone shows are impressive, but they are not mere pageantry. We may not be far from a world in which each drone is as small as a housefly, but lethal, and controlled by an intelligence greater than yours or mine.

Nor are their efforts restricted to military tech. Only recently, DeepSeek, a tiny Chinese company with no more than 200 employees, launched a free service that rivals its vastly more expensive American rivals. Not only that, but they published their entire source code online for free. Engineers are working round the clock to figure out how they did what they did. Possibly, the genius of the project lay in the unpublished training data, rather than the model itself. Time will tell. But regardless, what they pulled off is remarkable and industry-shattering. Who knows what other lightning advances are around the corner?

As China races forward, America scrambles to keep its competitive advantage. Vast American tech resources are nothing to be sneered at; even with DeepSeek’s great leap forward, OpenAI retains the title of premier large language model. Indeed, if the news about its o3 model is true, then we’re a few months away from accessing AI with more ability in most fields than doctorate holders in those same fields.

Washington certainly has no intention of falling behind. President Trump has revoked Biden’s AI safety orders. The full might of American tech may soon be unleashed. This may lead to wealth beyond our wildest imaginings. Whatever you spend on doctors, lawyers, accountants – information merchants of every kind – might soon be available for a few pennies worth of compute. The physical world of manufacturing would not be far behind. Perhaps we’ll all soon be at the beach while AI does all the work.

Or perhaps we will create AI superintelligence without adequate safeguards, and it will ignite the sky in a literal way that will make every fundamentalist feel vindicated for a very short period of time.

Biden’s feebleness, his criminal senescence, deserves all the mockery it got. But in this particular case we ought to be careful about celebrating the way Trump is unleashing American industry and innovation.

The problems do not end there. Any world in which there is a clear, looming threat of Armageddon will also be one where various veiled tyrants present themselves as solutions. Just as global Communism presented itself as the solution to international nuclear conflict, so too will leftists of one stripe or another claim that unless they are put in charge of the whole world, we’ll all be consumed by AI.

Fortunately, there is at least one voice near the president that knows what a dangerous game we’re playing. Elon Musk has frequently warned of how serious a risk AI poses to humanity’s future. In Part 1, I cited a paper from early 2023 that claimed the average AI researcher thought the chances of AI destroying all of humanity were roughly 10%. But early 2023 is an epoch ago in AI-research-time. Musk now puts the probability closer to 30%. He’s an optimist; the average AI researcher now says that our “p-doom”, the probability of our destruction, is at about 40%.

The People’s Republic of China doesn’t want the world to end either, and it may be that Xi’s advisors are as clued into this risk as Trump’s are. If so, it’s absolutely imperative that Xi and Trump begin laying the groundwork for some kind of non-tyrannical cooperation. For the reasons I laid out in Part 2, I think the chances of this working are not good at all. But it’s certainly worth a try, and soon. Perhaps the master deal-maker can find a way to save us all from our technological hubris, or at least delay our doom.

As private citizens, the best thing we can do is to keep the AI debate from becoming too party-polarized. Most people think along political tribal lines. There’s nothing wrong with this, since those who agree with a person on most issues now are likely to agree on most issues in the future. Tribalism here is a very bad idea. Democrats have exactly as much interest as Republicans in not destroying the world. If we end up in a nation where either party takes a feckless, greedy attitude towards AI research, then it is just a matter of time until we find out exactly how real the AI risk was.

I, for one, don’t intend to die squabbling with my neighbors. It’s time for the serious adults in both parties to put down their arms and find a path forward towards some kind of cooperation with China before it’s too late.