Memory-aligned Knowledge Graph for Clinically Accurate Radiology Image Report Generation

Sixing Yan

doi:10.18653/v1/2022.bionlp-1.11

Memory-aligned Knowledge Graph for Clinically Accurate Radiology Image Report Generation

Sixing Yan^*

^*Corresponding author for this work

Department of Computer Science

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

13 Citations (Scopus)

Abstract

Automatic generating the clinically accurate radiology report from X-ray images is important but challenging. The identification of multi-grained abnormal regions in image and corresponding abnormalities is difficult for data-driven neural models. In this work, we introduce a Memory-aligned Knowledge Graph (MaKG) of clinical abnormalities to better learn the visual patterns of abnormalities and their relationships by integrating it into a deep model architecture for the report generation. We carry out extensive experiments and show that the proposed MaKG deep model can improve the clinical accuracy of the generated reports.

Original language	English
Title of host publication	BioNLP 2022 @ ACL 2022 - Proceedings of the 21st Workshop on Biomedical Language Processing
Editors	Dina Demner-Fushman, Kevin Bretonnel Cohen, Sophia Ananiadou, Junichi Tsujii
Publisher	Association for Computational Linguistics (ACL)
Pages	116-122
Number of pages	7
ISBN (Electronic)	9781955917278
DOIs	https://doi.org/10.18653/v1/2022.bionlp-1.11
Publication status	Published - May 2022
Event	21st Workshop on Biomedical Language Processing, BioNLP 2022 at the Association for Computational Linguistics Conference, ACL 2022 - Dublin, Ireland Duration: 26 May 2022 → …

Publication series

Name	Proceedings of the Workshop on Biomedical Language Processing

Conference

Conference	21st Workshop on Biomedical Language Processing, BioNLP 2022 at the Association for Computational Linguistics Conference, ACL 2022
Country/Territory	Ireland
City	Dublin
Period	26/05/22 → …

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 3 Good Health and Well-being
SDG 9 Industry, Innovation, and Infrastructure

Access to Document

10.18653/v1/2022.bionlp-1.11

Cite this

Yan, S. (2022). Memory-aligned Knowledge Graph for Clinically Accurate Radiology Image Report Generation. In D. Demner-Fushman, K. B. Cohen, S. Ananiadou, & J. Tsujii (Eds.), BioNLP 2022 @ ACL 2022 - Proceedings of the 21st Workshop on Biomedical Language Processing (pp. 116-122). (Proceedings of the Workshop on Biomedical Language Processing). Association for Computational Linguistics (ACL). https://doi.org/10.18653/v1/2022.bionlp-1.11

Yan, Sixing. / Memory-aligned Knowledge Graph for Clinically Accurate Radiology Image Report Generation. BioNLP 2022 @ ACL 2022 - Proceedings of the 21st Workshop on Biomedical Language Processing. editor / Dina Demner-Fushman ; Kevin Bretonnel Cohen ; Sophia Ananiadou ; Junichi Tsujii. Association for Computational Linguistics (ACL), 2022. pp. 116-122 (Proceedings of the Workshop on Biomedical Language Processing).

@inproceedings{38bdf1247afb4bf6b487fae452e3aea9,

title = "Memory-aligned Knowledge Graph for Clinically Accurate Radiology Image Report Generation",

abstract = "Automatic generating the clinically accurate radiology report from X-ray images is important but challenging. The identification of multi-grained abnormal regions in image and corresponding abnormalities is difficult for data-driven neural models. In this work, we introduce a Memory-aligned Knowledge Graph (MaKG) of clinical abnormalities to better learn the visual patterns of abnormalities and their relationships by integrating it into a deep model architecture for the report generation. We carry out extensive experiments and show that the proposed MaKG deep model can improve the clinical accuracy of the generated reports.",

author = "Sixing Yan",

note = "Publisher Copyright: {\textcopyright} 2022 Association for Computational Linguistics.; 21st Workshop on Biomedical Language Processing, BioNLP 2022 at the Association for Computational Linguistics Conference, ACL 2022 ; Conference date: 26-05-2022",

year = "2022",

month = may,

doi = "10.18653/v1/2022.bionlp-1.11",

language = "English",

series = "Proceedings of the Workshop on Biomedical Language Processing",

publisher = "Association for Computational Linguistics (ACL)",

pages = "116--122",

editor = "Dina Demner-Fushman and Cohen, \{Kevin Bretonnel\} and Sophia Ananiadou and Junichi Tsujii",

booktitle = "BioNLP 2022 @ ACL 2022 - Proceedings of the 21st Workshop on Biomedical Language Processing",

address = "Australia",

}

Yan, S 2022, Memory-aligned Knowledge Graph for Clinically Accurate Radiology Image Report Generation. in D Demner-Fushman, KB Cohen, S Ananiadou & J Tsujii (eds), BioNLP 2022 @ ACL 2022 - Proceedings of the 21st Workshop on Biomedical Language Processing. Proceedings of the Workshop on Biomedical Language Processing, Association for Computational Linguistics (ACL), pp. 116-122, 21st Workshop on Biomedical Language Processing, BioNLP 2022 at the Association for Computational Linguistics Conference, ACL 2022, Dublin, Ireland, 26/05/22. https://doi.org/10.18653/v1/2022.bionlp-1.11

Memory-aligned Knowledge Graph for Clinically Accurate Radiology Image Report Generation. / Yan, Sixing.
BioNLP 2022 @ ACL 2022 - Proceedings of the 21st Workshop on Biomedical Language Processing. ed. / Dina Demner-Fushman; Kevin Bretonnel Cohen; Sophia Ananiadou; Junichi Tsujii. Association for Computational Linguistics (ACL), 2022. p. 116-122 (Proceedings of the Workshop on Biomedical Language Processing).

Research output: Chapter in book/report/conference proceeding › Conference proceeding › peer-review

TY - GEN

T1 - Memory-aligned Knowledge Graph for Clinically Accurate Radiology Image Report Generation

AU - Yan, Sixing

PY - 2022/5

Y1 - 2022/5

N2 - Automatic generating the clinically accurate radiology report from X-ray images is important but challenging. The identification of multi-grained abnormal regions in image and corresponding abnormalities is difficult for data-driven neural models. In this work, we introduce a Memory-aligned Knowledge Graph (MaKG) of clinical abnormalities to better learn the visual patterns of abnormalities and their relationships by integrating it into a deep model architecture for the report generation. We carry out extensive experiments and show that the proposed MaKG deep model can improve the clinical accuracy of the generated reports.

AB - Automatic generating the clinically accurate radiology report from X-ray images is important but challenging. The identification of multi-grained abnormal regions in image and corresponding abnormalities is difficult for data-driven neural models. In this work, we introduce a Memory-aligned Knowledge Graph (MaKG) of clinical abnormalities to better learn the visual patterns of abnormalities and their relationships by integrating it into a deep model architecture for the report generation. We carry out extensive experiments and show that the proposed MaKG deep model can improve the clinical accuracy of the generated reports.

UR - https://www.scopus.com/pages/publications/85143263380

U2 - 10.18653/v1/2022.bionlp-1.11

DO - 10.18653/v1/2022.bionlp-1.11

M3 - Conference proceeding

AN - SCOPUS:85143263380

T3 - Proceedings of the Workshop on Biomedical Language Processing

SP - 116

EP - 122

BT - BioNLP 2022 @ ACL 2022 - Proceedings of the 21st Workshop on Biomedical Language Processing

A2 - Demner-Fushman, Dina

A2 - Cohen, Kevin Bretonnel

A2 - Ananiadou, Sophia

A2 - Tsujii, Junichi

PB - Association for Computational Linguistics (ACL)

T2 - 21st Workshop on Biomedical Language Processing, BioNLP 2022 at the Association for Computational Linguistics Conference, ACL 2022

Y2 - 26 May 2022

ER -

Yan S. Memory-aligned Knowledge Graph for Clinically Accurate Radiology Image Report Generation. In Demner-Fushman D, Cohen KB, Ananiadou S, Tsujii J, editors, BioNLP 2022 @ ACL 2022 - Proceedings of the 21st Workshop on Biomedical Language Processing. Association for Computational Linguistics (ACL). 2022. p. 116-122. (Proceedings of the Workshop on Biomedical Language Processing). doi: 10.18653/v1/2022.bionlp-1.11

Memory-aligned Knowledge Graph for Clinically Accurate Radiology Image Report Generation

Abstract

Publication series

Conference

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this