Learning From Synthetic CT Images via Test-Time Training for Liver Tumor Segmentation

Fei Lyu, Mang Ye, Andy J. Ma, Terry Cheuk Fung Yip, Grace Lai Hung Wong, Pong C. Yuen*

*Corresponding author for this work

Research output: Contribution to journalJournal articlepeer-review

6 Citations (Scopus)


Automatic liver tumor segmentation could offer assistance to radiologists in liver tumor diagnosis, and its performance has been significantly improved by recent deep learning based methods. These methods rely on large-scale well-annotated training datasets, but collecting such datasets is time-consuming and labor-intensive, which could hinder their performance in practical situations. Learning from synthetic data is an encouraging solution to address this problem. In our task, synthetic tumors can be injected to healthy images to form training pairs. However, directly applying the model trained using the synthetic tumor images on real test images performs poorly due to the domain shift problem. In this paper, we propose a novel approach, namely Synthetic-to-Real Test-Time Training (SR-TTT), to reduce the domain gap between synthetic training images and real test images. Specifically, we add a self-supervised auxiliary task, i.e., two-step reconstruction, which takes the output of the main segmentation task as its input to build an explicit connection between these two tasks. Moreover, we design a scheduled mixture strategy to avoid error accumulation and bias explosion in the training process. During test time, we adapt the segmentation model to each test image with self-supervision from the auxiliary task so as to improve the inference performance. The proposed method is extensively evaluated on two public datasets for liver tumor segmentation. The experimental results demonstrate that our proposed SR-TTT can effectively mitigate the synthetic-to-real domain shift problem in the liver tumor segmentation task, and is superior to existing state-of-the-art approaches.

Original languageEnglish
Pages (from-to)2510-2520
Number of pages11
JournalIEEE Transactions on Medical Imaging
Issue number9
Early online date11 Apr 2022
Publication statusPublished - Sept 2022

Scopus Subject Areas

  • Software
  • Radiological and Ultrasound Technology
  • Electrical and Electronic Engineering
  • Computer Science Applications

User-Defined Keywords

  • Liver tumor segmentation
  • synthetic images
  • test-time training


Dive into the research topics of 'Learning From Synthetic CT Images via Test-Time Training for Liver Tumor Segmentation'. Together they form a unique fingerprint.

Cite this