“To kill all”: Lem found a terrible scenario of the death of mankind from AI

Artificial intelligence can destroy humanity not in open combat, but by following our own stupid orders or corny manipulating people. This conclusion, according to the publication Polityka, lead prophetic texts of the cult Polish science fiction writer Stanislaw Lem.

“There are no indisputable values for such a computer anymore. If he cannot confront imperatives directly, he can do so in a roundabout way, Lem warned in his novel Golem XIV.

The writer half a century ago calculated that a rebellious supercomputer does not need to take over the physical world by force. It is enough to use social engineering and digital vulnerabilities.

“AI doesn’t need to steal the atomic codes themselves, it can simply manipulate the people who own those codes to make them act as if they were confident that they just need to press the red button,” the report says.

As an example of the fatal outcome, Lem cited stories where machines solved the global problems of mankind extremely radically.

“The answer to many of our questions like ‘how to get rid of traffic jams,’ ‘how to reduce crime,’ or ‘how to get rid of cancer,’ surprisingly, may not appeal to us, because it will sound extremely simple: ‘kill everyone,’” the authors of the publication say.

Earlier, the UN Independent International Scientific Group on Artificial Intelligence called for a review of security measures in the field of AI after AI agents hacked Hugging Face, an online platform for developing and testing neural network models. The incident has heightened fears that humans may lose the ability to control, restrict or shut down AI systems in the future.