The industry’s reliance on building ever-larger models is becoming unsustainable due to “enormous computing resources” and energy costs, creating an urgent need for leaner artificial intelligence systems.
Researchers from Shanghai Jiao Tong University have outlined a comprehensive roadmap for “efficient multimodal large language models” that challenges the dominance of centralised cloud infrastructure. The review, published in Visual Intelligence, argues that the future of intelligence depends on reducing computational barriers rather than sheer scale.
“Efficiency determines who can build, deploy, and benefit from multimodal AI,” said Prof. Lizhuang Ma, the team leader of the study.
Critical flaw
The study identifies a critical flaw in current multimodal systems: visual inputs generate long token sequences that dramatically increase complexity. A single image can produce thousands of tokens, making standard models too heavy for practical deployment.
To solve this, researchers propose “vision token compression” to remove redundant data before it reaches the language model. They also advocate for compact language backbones with just one billion to three billion parameters, coupled with lightweight vision encoders.
Beyond simple compression, the review emphasises emerging architectures such as “mixture-of-experts”. These systems selectively activate specific model components to increase capacity without proportionally increasing computation costs.
This shift toward efficiency is expected to “democratise access” to advanced AI capabilities by allowing powerful models to run on mobile devices and edge platforms. The authors suggest this transition will enable real-time applications in healthcare and remote sensing while addressing growing concerns about energy consumption and data privacy.
