Skip to main navigation Skip to search Skip to main content

TADM: Unified Pre-trained Framework for EEG-to-fNIRS Cross-Modal Generation in BCI Decoding

  • Bin Yuan*
  • , Yisheng Li
  • , Shanshan Wang
  • , Zhiguo Zhang
  • , Zhi An Huang
  • , Hongzhi Kuai
  • , Xingming Zhao
  • , Ning Zhong
  • , Michael Kwok Po Ng
  • , Kim Fung Tsang
  • , Yishan Wang
  • , Shuqiang Wang
  • *Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

Abstract

Employing electroencephalography (EEG) and functional near-infrared spectroscopy(fNIRS) signals jointly is pivotal for advancing brain-computer interface (BCI) decoding, yet physical clashes between EEG and fNIRS channels prevent true simultaneous capture. Since EEG is cheap and easy to collect, we can sidestep the hardware conflict by deriving the missing fNIRS traces from the recorded EEG. The core challenge is to learn this mapping in a way that transfers across subjects, tasks and recording rigs without retraining. To address this issue, this work proposes the Pretrained Unified Framework: Topology-Adaptive Diffusion Model(TADM) for EEG to fNIRS Cross-Modal Generation in BCI-decoding. Trained on larger and more diverse datasets, TADM aims to enhance feature extraction capabilities and construct a more generalizable modality mapping mechanism. To address the heterogeneity between EEG and fNIRS signals, the model adopts a unified data alignment strategy for effective alignment and incorporates a pretrained model that leverages the geometric configuration of EEG and fNIRS sensors to construct a unified representation. The final cross-modal generation is achieved via a conditional diffusion process that transforms EEG latent representations into corresponding fNIRS latent representations. Experimental results demonstrate that TADM exhibits strong robustness and broad adaptability across diverse task scenarios and sensor configurations.

Original languageEnglish
Number of pages12
JournalIEEE Transactions on Consumer Electronics
DOIs
Publication statusE-pub ahead of print - 9 Apr 2026

User-Defined Keywords

  • Cross-modal generation
  • electroencepha-lography (EEG)
  • functional near-infrared spectroscopy (fNIRS)
  • motor imagery (MI)
  • pretrained model

Fingerprint

Dive into the research topics of 'TADM: Unified Pre-trained Framework for EEG-to-fNIRS Cross-Modal Generation in BCI Decoding'. Together they form a unique fingerprint.

Cite this