An Extended Active Learning Approach to Multiverse Analysis: Predictions of Latent Variables from Graph Theory Measures of the Human Connectome and Their Direct Replication

Daniel Kristanto*, Carsten Giesing, Merle Marek, Changsong Zhou, Stefan Debener, Christiane Thiel, Andrea Hildebrandt

*Corresponding author for this work

Research output: Contribution to journalConference articlepeer-review

1 Citation (Scopus)

Abstract

Multiverse analysis has been proposed as a powerful technique to disclose the large number of degrees of freedom in data preprocessing and analysis that strongly contribute to the current replication crisis in science. However, in the field of imaging neuroscience, where multidimensional, complex and noisy data are measured, multiverse analysis may be computationally infeasible. The number of possible forking paths given by different methodological decisions and analytical choices is immense. Recently, Dafflon et al. (2022) proposed an active learning approach as an alternative to exhaustively exploring all forking paths. Here, we aimed to extend their active learning pipeline by integrating latent underlying variables which are not directly observable. The extension to latent outcomes is particularly valuable for computational psychiatry and neurocognitive psychology, where latent traits are conceptualized as common cause of a variety of observable neural and behavioral symptoms. To illustrate our approach and to test its direct replicability, we analyzed the individual organization and topology of functional brain networks of two relatively large samples from the ABCD study dataset (N = 1491) and HCP dataset (N = 833). Graph-theoretical parameters that take into account both brain-wide and region-specific network properties were used as predictors of a latent variable reflecting general cognition. Our results demonstrate the ability of the extended method to selectively explore the multiverse when predicting a latent variable. First, the low-dimensional space created with the proposed approach was able to cluster the forking paths according to their similarity. Second, the active learning approach successfully estimated the prediction performance of all pipelines in both datasets. To interactively explore the multiverse of results, we developed a Shiny app to visualize the predictive accuracy resulting from each forking path and to illustrate the similarity between pipelines created by different combinations of data processing choice. The code for active learning and the app are available at the Github repository ExtendedAL.

Original languageEnglish
Article numberJ962E0F53
Number of pages13
JournalBrainiacs Journal of Brain Imaging And Computing Sciences
Volume4
Issue number2
DOIs
Publication statusPublished - 21 Dec 2023
Event2023 Guardians Workshop, Guardians 2023 - Ladera Ranch, United States
Duration: 9 Oct 2023 → …
https://brainiacsjournal.org/Archive/Issue/2023/4/2
https://ieeexplore.ieee.org/xpl/conhome/10473739/proceeding

Scopus Subject Areas

  • Safety, Risk, Reliability and Quality
  • Cognitive Neuroscience
  • Library and Information Sciences
  • Artificial Intelligence
  • Information Systems

User-Defined Keywords

  • active learning
  • latent variable modeling
  • Multiverse analysis
  • Shiny application

Cite this