UniMind: Unleashing the Power of LLMs for Unified Multi-Task Brain Decoding

Weiheng Lu1,2*, Zhouheng Yao1*, Jiamin Wu1,3†, Pengyu Zhu1, Yuchen Zhou1, Weijian Mai1, Qihao Zheng1, Wanli Ouyang1,3, Chunfeng Song1
1Shanghai Artificial Intelligence Laboratory, 2Peking University, 3The Chinese University of Hong Kong
*Indicates Equal Contribution, †Indicates Correspondence Authors
Paper Code
10public EEG datasets
5brain decoding task domains
929Kinstruction-tuning samples
11%gain over multi-task SOTA
UniMind Framework Overview

Overview of UniMind framework architecture.

Abstract

Decoding human brain activity from electroencephalography (EEG) signals is a central challenge for neuroscience and artificial intelligence, with applications in mental-state assessment, clinical monitoring, and human-machine interaction. Existing EEG foundation models often struggle with task heterogeneity and typically require task-specific tuning to achieve strong downstream performance. UniMind is a general-purpose EEG foundation model for unified multi-task brain decoding. It leverages large language models to comprehend complex neural patterns by aligning EEG representations with language-model hidden spaces. The model introduces a Neuro-Language Connector to bridge the neural-language modality gap and a Task-aware Query Selection module to dynamically generate task-adaptive query tokens for heterogeneous EEG tasks. Across ten public EEG datasets covering five brain-decoding domains, UniMind achieves strong multi-task decoding performance and provides neuroscientific insights into functional correlations across tasks through query-selection visualizations.

Method

UniMind Method

Overview of the UniMind architecture. Raw EEG signals are encoded into EEG embeddings $\boldsymbol{E}$, which are processed by the Task-aware Query Selector to extract task-adaptive queries. These queries are combined with static queries and jointly processed with $\boldsymbol{E}$ by the Neuro-Language Connector, which aligns spatio-temporal neural features with the LLM's semantic space. The resulting embeddings, together with task prompts, guide the LLM to generate text output.

Results

Performance comparison across EEG datasets. Underlined values are the best among multi-task models, and bold values are the best overall.

Results

Neuroscientific Insights

Task-aware query selection can also be used as an analysis tool for discovering functional correlations among EEG tasks.

Task Correlation

Similar decoding tasks tend to select similar query patterns, suggesting reusable neural mechanisms across related datasets.

Knowledge Sharing

Task-adaptive routing promotes transfer across compatible tasks while reducing interference from heterogeneous task formats.

Query Visualization

Query-selection statistics provide an interpretable window into how a unified model organizes neural functions across tasks.

BibTeX

@article{lu2025unimind,
  title   = {UniMind: Unleashing the Power of LLMs for Unified Multi-Task Brain Decoding},
  author  = {Lu, Weiheng and Yao, Zhouheng and Wu, Jiamin and Zhu, Pengyu and Zhou, Yuchen and Mai, Weijian and Zheng, Qihao and Ouyang, Wanli and Song, Chunfeng},
  journal = {arXiv preprint arXiv:2506.18962},
  year    = {2025},
  url     = {https://arxiv.org/abs/2506.18962}
}