Does a Picture Hurt More Than a Thousand Words? A Systematic Examination of Multimodal Hate Speech in Hong Kong

Hateful rhetoric is a rampant problem in today's online environments. Despite a growing body of research on online hate speech, including understanding it and its impact, there are notable gaps. First, previous research has focused primarily on hate speech as expressed in text, neglecting the critical role of visuals in this phenomenon. Given the multimodal nature of social media platforms, failing to consider multimodal hate speech—which combines text and images to express hate—undermines attempts at understanding and hinders efforts to develop effective solutions. Second, most empirical
research on multimodal hate speech has been conducted in Western countries, raising questions about the generalizability of these findings to other contexts. Third, the majority of experimental studies on the impact of multimodal hate speech have employed single-exposure, one-shot designs, providing limited insights into the duration of the effects described and the potential consequences of repeated exposure. Finally, there is a dearth of research on strategies to combat multimodal hate speech, so it remains unclear whether approaches that have proven effective in countering textual hate speech also work for multimodal hate speech.

Therefore, the current project aims to address these critical research voids by systematically examining the characteristics, effects, and countermeasures of multimodal hate speech in the context of Hong Kong. Using a multiple-method approach and building upon the research team's previous work on examining hate speech, we will conduct a series of studies to gain a deeper conceptual and empirical understanding of multimodal hate speech. Specifically, Study 1 will involve a quantitative content analysis to explore and identify the characteristics of multimodal hate speech on social media. Study 2 will be a longitudinal experiment to investigate the short- and long-term effects of exposure to multimodal hate speech on prejudicial attitudes and discriminatory behaviors toward targeted groups. Study 3 will experimentally examine the effectiveness of counter-speech (e.g., providing facts to refute hateful claims) in mitigating the negative consequences of multimodal hate speech.

As one of the initial attempts to conduct a holistic investigation of multimodal hate speech, this project will contribute unique theoretical insights into this phenomenon by linking hate speech research with relevant and valuable resources in the visual communication literature. Additionally, the results will provide practical guidance for policymakers and those involved in the creation and management of online platforms to address the potential harm of multimodal hateful content and foster more respectful and inclusive online environments.