Evaluation of the reliability and usability of CARE-Radiology: A descriptive-analytic study

Mengshu Wang; Xufei Luo; Janne Estill; Karen Spruyt; Ryo Kurokawa; Nav Persaud; Yasuteru Shimamura; Holly Raison; Paolo Niccolò Franco; Cesare Maino; Hussein Elkhayat; Rehab A. Galal; Daisuke Kimura; Shingo Omata; Sergey A. Ryzhkin; Timur R. Izmailov; Rustem A. Bashirov; Zhaoxiang Bian; Jinhui Tian; Junqiang Lei

doi:10.1111/jebm.12638

Evaluation of the reliability and usability of CARE-Radiology: A descriptive-analytic study

Mengshu Wang, Xufei Luo, Janne Estill, Karen Spruyt, Ryo Kurokawa, Nav Persaud, Yasuteru Shimamura, Holly Raison, Paolo Niccolò Franco, Cesare Maino, Hussein Elkhayat, Rehab A. Galal, Daisuke Kimura, Shingo Omata, Sergey A. Ryzhkin, Timur R. Izmailov, Rustem A. Bashirov, Zhaoxiang Bian, Jinhui Tian, Junqiang Lei^*

^*Corresponding author for this work

Chinese Medicine - Clinical Division

Research output: Contribution to journal › Journal article › peer-review

Abstract

Aim: The study aimed to evaluate the reliability and usability of the CARE-Radiology checklist in assessing radiological case reports and provide a basis for its broader adoption and optimization.

Methods: Ten randomly selected radiological case reports published in scientific journals in 2020 were evaluated using the CARE-Radiology checklist. Twenty-six experts from 10 countries were invited to independently assess all ten reports. The reliability of the checklist was measured using Fleiss' Kappa, and Cronbach's alpha coefficient. Usability was evaluated by recording the time taken to complete the assessments and requesting the evaluators to rate each item on a Likert scale for its easiness of use.

Results: The median time for evaluating one radiological case report was 15 min. The overall agreement among evaluators showed moderate reliability with a Kappa value of 0.47 and a Cronbach's alpha of 0.51. The mean compliance rate for the items of CARE-Radiology was 61.8%, with some items exceeding 90% compliance. Items related to abstracts and keywords had the lowest compliance rates. The evaluators found most items easy to understand, with a few exceptions.

Conclusions: The CARE-Radiology checklist is relatively easy for researchers to use and understand. Continuous feedback is necessary for future revisions and updates, to enhance the effectiveness of the checklist, and to improve user experience.

Original language	English
Pages (from-to)	597-603
Number of pages	7
Journal	Journal of Evidence-Based Medicine
Volume	17
Issue number	3
DOIs	https://doi.org/10.1111/jebm.12638
Publication status	Published - Sept 2024

Scopus Subject Areas

Health Policy

User-Defined Keywords

CARE
case report
radiology
reliability
usability

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

Access to Document

10.1111/jebm.12638

Cite this

Wang, M., Luo, X., Estill, J., Spruyt, K., Kurokawa, R., Persaud, N., Shimamura, Y., Raison, H., Franco, P. N., Maino, C., Elkhayat, H., Galal, R. A., Kimura, D., Omata, S., Ryzhkin, S. A., Izmailov, T. R., Bashirov, R. A., Bian, Z., Tian, J., & Lei, J. (2024). Evaluation of the reliability and usability of CARE-Radiology: A descriptive-analytic study. Journal of Evidence-Based Medicine, 17(3), 597-603. https://doi.org/10.1111/jebm.12638

@article{b3e2af66beff4466b16049b1ab2dfad2,

title = "Evaluation of the reliability and usability of CARE-Radiology: A descriptive-analytic study",

abstract = "Aim: The study aimed to evaluate the reliability and usability of the CARE-Radiology checklist in assessing radiological case reports and provide a basis for its broader adoption and optimization.Methods: Ten randomly selected radiological case reports published in scientific journals in 2020 were evaluated using the CARE-Radiology checklist. Twenty-six experts from 10 countries were invited to independently assess all ten reports. The reliability of the checklist was measured using Fleiss' Kappa, and Cronbach's alpha coefficient. Usability was evaluated by recording the time taken to complete the assessments and requesting the evaluators to rate each item on a Likert scale for its easiness of use.Results: The median time for evaluating one radiological case report was 15 min. The overall agreement among evaluators showed moderate reliability with a Kappa value of 0.47 and a Cronbach's alpha of 0.51. The mean compliance rate for the items of CARE-Radiology was 61.8%, with some items exceeding 90% compliance. Items related to abstracts and keywords had the lowest compliance rates. The evaluators found most items easy to understand, with a few exceptions.Conclusions: The CARE-Radiology checklist is relatively easy for researchers to use and understand. Continuous feedback is necessary for future revisions and updates, to enhance the effectiveness of the checklist, and to improve user experience.",

keywords = "CARE, case report, radiology, reliability, usability",

author = "Mengshu Wang and Xufei Luo and Janne Estill and Karen Spruyt and Ryo Kurokawa and Nav Persaud and Yasuteru Shimamura and Holly Raison and Franco, {Paolo Niccol{\`o}} and Cesare Maino and Hussein Elkhayat and Galal, {Rehab A.} and Daisuke Kimura and Shingo Omata and Ryzhkin, {Sergey A.} and Izmailov, {Timur R.} and Bashirov, {Rustem A.} and Zhaoxiang Bian and Jinhui Tian and Junqiang Lei",

note = "Publisher Copyright: {\textcopyright} 2024 Chinese Cochrane Center, West China Hospital of Sichuan University and John Wiley & Sons Australia, Ltd.",

year = "2024",

month = sep,

doi = "10.1111/jebm.12638",

language = "English",

volume = "17",

pages = "597--603",

journal = "Journal of Evidence-Based Medicine",

issn = "1756-5383",

publisher = "Wiley-Blackwell Publishing Ltd",

number = "3",

}

Wang, M, Luo, X, Estill, J, Spruyt, K, Kurokawa, R, Persaud, N, Shimamura, Y, Raison, H, Franco, PN, Maino, C, Elkhayat, H, Galal, RA, Kimura, D, Omata, S, Ryzhkin, SA, Izmailov, TR, Bashirov, RA, Bian, Z, Tian, J & Lei, J 2024, 'Evaluation of the reliability and usability of CARE-Radiology: A descriptive-analytic study', Journal of Evidence-Based Medicine, vol. 17, no. 3, pp. 597-603. https://doi.org/10.1111/jebm.12638

TY - JOUR

T1 - Evaluation of the reliability and usability of CARE-Radiology

T2 - A descriptive-analytic study

AU - Wang, Mengshu

AU - Luo, Xufei

AU - Estill, Janne

AU - Spruyt, Karen

AU - Kurokawa, Ryo

AU - Persaud, Nav

AU - Shimamura, Yasuteru

AU - Raison, Holly

AU - Franco, Paolo Niccolò

AU - Maino, Cesare

AU - Elkhayat, Hussein

AU - Galal, Rehab A.

AU - Kimura, Daisuke

AU - Omata, Shingo

AU - Ryzhkin, Sergey A.

AU - Izmailov, Timur R.

AU - Bashirov, Rustem A.

AU - Bian, Zhaoxiang

AU - Tian, Jinhui

AU - Lei, Junqiang

PY - 2024/9

Y1 - 2024/9

N2 - Aim: The study aimed to evaluate the reliability and usability of the CARE-Radiology checklist in assessing radiological case reports and provide a basis for its broader adoption and optimization.Methods: Ten randomly selected radiological case reports published in scientific journals in 2020 were evaluated using the CARE-Radiology checklist. Twenty-six experts from 10 countries were invited to independently assess all ten reports. The reliability of the checklist was measured using Fleiss' Kappa, and Cronbach's alpha coefficient. Usability was evaluated by recording the time taken to complete the assessments and requesting the evaluators to rate each item on a Likert scale for its easiness of use.Results: The median time for evaluating one radiological case report was 15 min. The overall agreement among evaluators showed moderate reliability with a Kappa value of 0.47 and a Cronbach's alpha of 0.51. The mean compliance rate for the items of CARE-Radiology was 61.8%, with some items exceeding 90% compliance. Items related to abstracts and keywords had the lowest compliance rates. The evaluators found most items easy to understand, with a few exceptions.Conclusions: The CARE-Radiology checklist is relatively easy for researchers to use and understand. Continuous feedback is necessary for future revisions and updates, to enhance the effectiveness of the checklist, and to improve user experience.

AB - Aim: The study aimed to evaluate the reliability and usability of the CARE-Radiology checklist in assessing radiological case reports and provide a basis for its broader adoption and optimization.Methods: Ten randomly selected radiological case reports published in scientific journals in 2020 were evaluated using the CARE-Radiology checklist. Twenty-six experts from 10 countries were invited to independently assess all ten reports. The reliability of the checklist was measured using Fleiss' Kappa, and Cronbach's alpha coefficient. Usability was evaluated by recording the time taken to complete the assessments and requesting the evaluators to rate each item on a Likert scale for its easiness of use.Results: The median time for evaluating one radiological case report was 15 min. The overall agreement among evaluators showed moderate reliability with a Kappa value of 0.47 and a Cronbach's alpha of 0.51. The mean compliance rate for the items of CARE-Radiology was 61.8%, with some items exceeding 90% compliance. Items related to abstracts and keywords had the lowest compliance rates. The evaluators found most items easy to understand, with a few exceptions.Conclusions: The CARE-Radiology checklist is relatively easy for researchers to use and understand. Continuous feedback is necessary for future revisions and updates, to enhance the effectiveness of the checklist, and to improve user experience.

KW - CARE

KW - case report

KW - radiology

KW - reliability

KW - usability

UR - http://www.scopus.com/inward/record.url?scp=85204387284&partnerID=8YFLogxK

UR - https://onlinelibrary.wiley.com/doi/10.1111/jebm.12638

U2 - 10.1111/jebm.12638

DO - 10.1111/jebm.12638

M3 - Journal article

AN - SCOPUS:85204387284

SN - 1756-5383

VL - 17

SP - 597

EP - 603

JO - Journal of Evidence-Based Medicine

JF - Journal of Evidence-Based Medicine

IS - 3

ER -

Evaluation of the reliability and usability of CARE-Radiology: A descriptive-analytic study

Abstract

Scopus Subject Areas

User-Defined Keywords

UN SDGs

Access to Document

Other files and links

Fingerprint

Cite this