Top-down generation of low-resolution representations improves visual perception and imagination

Zedong Bi*, Haoran Li, Liang Tian*

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

Abstract

Perception or imagination requires top-down signals from high-level cortex to primary visual cortex (V1) to reconstruct or simulate the representations bottom-up stimulated by the seen images. Interestingly, top-down signals in V1 have lower spatial resolution than bottom-up representations. It is unclear why the brain uses low-resolution signals to reconstruct or simulate high-resolution representations. By modeling the top-down pathway of the visual system using the decoder of a variational auto-encoder (VAE), we reveal that low-resolution top-down signals can better reconstruct or simulate the information contained in the sparse activities of V1 simple cells, which facilitates perception and imagination. This advantage of low-resolution generation is related to facilitating high-level cortex to form geometry-respecting representations observed in experiments. Furthermore, we present two findings regarding this phenomenon in the context of AI-generated sketches, a style of drawings made of lines. First, we found that the quality of the generated sketches critically depends on the thickness of the lines in the sketches: thin-line sketches are harder to generate than thick-line sketches. Second, we propose a technique to generate high-quality thin-line sketches: instead of directly using original thin-line sketches, we use blurred sketches to train VAE or GAN (generative adversarial network), and then infer the thin-line sketches from the VAE- or GAN-generated blurred sketches. Collectively, our work suggests that low-resolution top-down generation is a strategy the brain uses to improve visual perception and imagination, which inspires new sketch-generation AI techniques.

Original languageEnglish
Pages (from-to)440-456
Number of pages17
JournalNeural Networks
Volume171
Early online date19 Dec 2023
DOIs
Publication statusE-pub ahead of print - 19 Dec 2023

Scopus Subject Areas

  • Cognitive Neuroscience
  • Artificial Intelligence

User-Defined Keywords

  • Generative model
  • Sketch generation
  • Visual system

Fingerprint

Dive into the research topics of 'Top-down generation of low-resolution representations improves visual perception and imagination'. Together they form a unique fingerprint.

Cite this