Research on Real-Time Privacy-Preserving Sign Language Translation and Production

According to official statistics, the number of people with hearing impairments in mainland China and Hong Kong is approximately 30 million, and this proportion of the population is increasing rapidly each year. Sign language is the primary medium for communication between people with hearing impairments and others. However, there is currently a severe shortage of sign language interpreters, and existing automated sign language translation tools face challenges such as high latency, low accuracy, and significant risks of privacy leaks. These issues greatly hinder effective and timely information exchange for the hearing-impaired community. To address these problems, this project aims to develop an open dataset encompassing a variety of sign language data through a crowdsourcing data collection mechanism. Additionally, it intends to integrate attention mechanisms with neural machine translation techniques to design a real-time high-accuracy sign language recognition system. Furthermore, through experimental analysis of various NLP neural network architectures, the project will explore efficient and precise real-time sign language generation models. By combining the characteristics of these sign language translation models with fully homomorphic encryption algorithms, it will develop a low-complexity, privacy-preserving solution for real-time sign language translation. Finally, by adapting to a heterogeneous computing platform, this project aims to build a real-time, secure, and private open computing platform for sign language translation, setting a social application model to provide high-quality sign language translation services for the hearing-impaired community. Furthermore, through the in-depth innovative cooperation of “One University, Two Campuses”, we will achieve the cross-regional in-depth integration of AI for Social Good for the deaf and hard-of-hearing group in the Guangdong-Hong Kong Greater Bay Area.