We use a few cookies

Essential cookies keep the library working. With your permission, we'd also like to use analytics to see what's useful and improve the experience. Read our Privacy Policy or Cookie Policy.

Skip to main content
Home/e-Resources/Artificial Intelligence & Machine Learning/Machine Translation from Signed to Spoken Languages: State of the Art and Challenges
PeriodicalsArtificial Intelligence & Machine Learning

Machine Translation from Signed to Spoken Languages: State of the Art and Challenges

arXiv2022

Checking your access to this record.

Abstract & Overview

Automatic translation from signed to spoken languages is an interdisciplinary research domain, lying on the intersection of computer vision, machine translation and linguistics. Nevertheless, research in this domain is performed mostly by computer scientists in isolation. As the domain is becoming increasingly popular - the majority of scientific papers on the topic of sign language translation have been published in the past three years - we provide an overview of the state of the art as well as some required background in the different related disciplines. We give a high-level introduction to sign language linguistics and machine translation to illustrate the requirements of automatic sign language translation. We present a systematic literature review to illustrate the state of the art in the domain and then, harking back to the requirements, lay out several challenges for future research. We find that significant advances have been made on the shoulders of spoken language machine translation research. However, current approaches are often not linguistically motivated or are not adapted to the different input modality of sign languages. We explore challenges related to the representation of sign language data, the collection of datasets, the need for interdisciplinary research and requirements for moving beyond research, towards applications. Based on our findings, we advocate for interdisciplinary research and to base future research on linguistic analysis of sign languages. Furthermore, the inclusion of deaf and hearing end users of sign language translation applications in use case identification, data collection and evaluation is of the utmost importance in the creation of useful sign language translation models. We recommend iterative, human-in-the-loop, design and development of sign language translation models.

Cite this record

Mathieu De Coster, Dimitar Shterionov, Mieke Van Herreweghe, Joni Dambre (2022). Machine Translation from Signed to Spoken Languages: State of the Art and Challenges. arXiv. https://library.shubhaarti.com/content/285c33d1-10ba-445e-aa8a-9459778bdf02

Copy in another style

Coster, M. D., Shterionov, D., Herreweghe, M. V., & Dambre, J. (2022). Machine Translation from Signed to Spoken Languages: State of the Art and Challenges. arXiv. https://library.shubhaarti.com/content/285c33d1-10ba-445e-aa8a-9459778bdf02
Author names are used as deposited, so check their order before you submit.

Metadata is harvested from open-access indexes and can be incomplete. Check a citation against the published version before relying on it — see our open access & attribution policy.