“The development of artificial intelligence (AI) is at a turning point,” says computer science professor Björn Ommer, and thousands of AI experts are listening to him. The AI models like the one behind the Chat-GPT chatbot have become bigger and bigger. Too big, says Ommer. Even with self-learning software, the “limits to growth” that the Club of Rome formulated for the entire planet 51 years ago have now been reached.
Ommer is a professor at Munich’s Ludwig Maximilians University and heads the Compvis group, which teaches computers to see. This Monday he will give the keynote speech on stage at the New Orleans Convention Center.
In front of him in the gigantic hall sit the people who are building the future. New Orleans is the AI capital of the world this week. The most important conference in the field, Neurips, is taking place. 13,000 people came, all elite universities and tech companies represented. The conference is considered the “Superbowl of artificial intelligence” and tickets for it were sold out within minutes.
Ommer is here to warn his colleagues. That when it comes to AI, everything is leading to a concentration of power. That Google, Microsoft and Amazon have the computing power, the data and the licenses to dominate the market. And there is only one thing that can help: the AI has to be so small that it fits on a smartphone.
The idealist Ommer is the thoughtful European counter-model to Sam Altman, the Open AI boss. He is known for the commercialization of his main work Chat-GPT, billion-dollar deals with Microsoft, lobbying and a lack of transparency when it comes to what data his AI was trained with.
Ommer’s speech is entitled “The Illusion of Scaling”, translated for non-computer scientists: Here someone wants to curb the megalomania of his own community. The Munich professor is a big name himself here, because his research group is responsible for one of the models that triggered the AI hype in 2022. Since then, the technology has occupied the economy, society and politics. Stable Diffusion is the name of the AI model from Ommer. The software can only create amazingly good images based on a text description. The image created by Stable Diffusion became iconic in August 2022 Astronaut riding a horse. And the program was early. Chat-GPT wasn’t released until three months later.
The professor would have had the opportunity to earn millions with his invention. The start-up Stability AI, which supported and distributed Ommer’s open source image program early on, was recently valued at one billion euros. Some of Ommer’s former employees now also work there. But he remained loyal to research. “I still find the research questions very interesting, and for me it’s about democratization, otherwise the imbalances between small and large people would have become even greater. In this respect, I have taken the right path,” he told the SZ before his speech.
Smart algorithms instead of huge algorithms
As a counter to the gigantomania of the rest of the AI industry, Ommer promotes the small one. The AI in your pocket. “A truly intelligent algorithm is not necessarily one that uses more computing power. The reverse should be true: smaller models should be the best.” Large models like GPT-4 would have to be used online, for example in the Microsoft cloud. On stage, he lectures his colleagues: Scaling – more computing power, more data, ever larger models – has been the answer to all the problems in the field for too long.
This development is coming to an end: “Many large language models have flattened out in the last few months.” Even the new version of Chat-GPT shows weaknesses, even though the company Open AI has invested a lot of money and computing power from its investor Microsoft. Ommer promotes smart algorithms that are less greedy for resources. “Stable Diffusion now runs on the Mac. You no longer need a supercomputer that costs millions of euros.”
The greed of the big AI companies for more data and more computing power is not only an ideological problem, but also a very practical one. The production of graphics chips necessary for AI can no longer begin to meet demand. The USA, for example, has already banned the export of modern AI chips to China as if they were weapons.
Ommer says that the large language models have not understood something fundamental about intelligence: that it is not abundance, but rather limitation that leads to intelligent solutions. Our ancestors would have developed their intelligence with sticks and stones and not in comfortable cars.
Even with AI, less is more. For example, the systems don’t need to know every detail about images. “They want to know the dog is fluffy, but they don’t want to know what every single hair looks like.” A photo of a white dog now lights up on the screen behind Ommer.
After the fluff, things get technical, appropriate to the proportion of computer scientists in the room. Ommer shares his team’s experiences with experts on how to shrink models. In terms of the efficiency of AI for smartphones. And in the spirit of democracy, for which AI is becoming a key technology.
Because Ommer is about more than a likeable “small but mighty”. He says: “The things that hold society together, like trust, don’t scale.” A diamond only exists because the carbon holds together due to the high pressure. If you replace these close connections with loose ones – such as direct contacts between people through purely online exchanges on social media – then the diamond becomes ashes.