Navigation auf uzh.ch

Suche

Department of Computational Linguistics Language, Technology and Accessibility

Sign Language Technologies

Pipeline for going from sign language to spoken language or vice versa, via translation, production, and recognition
Bild in Detailansicht öffnen

Sign languages are characterized by the use of multiple production modalities, making them inherently multimodal: In addition to the more familiar manual features – the activity of the hands and arms –, there are non-manual features, i.e. static and dynamic features of the upper body, head and face (eyebrows, gaze, eyelids, nose, mouth). These features are equally significant from a linguistic point of view. Incorporating non-manual features into automatic sign language processing (sign language recognition, translation, and synthesis) forms one of the research challenges we are dealing with. In addition to the core processing tasks, our group also investigates computer-assisted sign language assessment.
    
Note that, contrary to what is sometimes reported in the media, the state of sign language technology is not such that, e.g., fully automatic sign language translation in an unrestricted domain would be possible.

Weiterführende Informationen

IICT logo showing an illustration face in black and blue

Inclusive Information and Communication Technologies (IICT)

More about Inclusive Information and Communication Technologies (IICT)

IICT is a four-year project with an overall budget of  12.3 million CHF supported by the Swiss Innovation Agency (Innosuisse) under its flagship scheme. The goal of the IICT flagship is to develop information and communication technologies (ICT) in the context of persons with disabilities. The flagship targets five applications: text simplification, sign language translation, sign language assessment, audio description, and spoken subtitles.
 

EASIER logo

Intelligent Automatic Sign Language Translation (EASIER)

More about Intelligent Automatic Sign Language Translation (EASIER)

EASIER aims to create a framework for barrier-free communication among deaf and hearing citizens across Europe by enabling users of European sign languages to use their preferred language to interact with hearing individuals.
 

SMILE web interface screenshot

Scalable Multimodal Sign Language Technology for Sign Language Learning and Assessment, Phase II (SMILE II)

More about Scalable Multimodal Sign Language Technology for Sign Language Learning and Assessment, Phase II (SMILE II)

SMILE II aims to research and build advanced technology for sign language learning. More precisely, the project builds on the groundwork of the SNSF Sinergia project SMILE I, which dealt with assessment of the manual activity of Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS) in isolated signs produced by L1 users and L2 learners. SMILE II extends this technology to continuous sign language assessment including both manual and non-manual components of signs so that a DSGS learner’s sentence-level production can be assessed in an automatic manner.

SMILE participants collage

Scalable Multimodal Sign Language Technology for Sign Language Learning and Assessment (SMILE I)

The goal of this project was to pioneer an assessment system for Swiss German Sign Language (Deutschschweizerische Gebärdensprache, DSGS) using automatic sign language recognition technology. To achieve this goal, the project used a multidisciplinary framework that followed two strands of research, one on sign language technology and one on sign assessment with a common link to sign language linguistics.
 

Mobile phone at a train station

Automatic Translation of German Train Announcements into Swiss German Sign Language (Trainslate)

More about Automatic Translation of German Train Announcements into Swiss German Sign Language (Trainslate)

In the Trainslate (=train+translate) project we developed a system that automatically translates German train announcements of the Swiss Federal Railways (Schweizerische Bundesbahnen, SBB) into Swiss German Sign Language. The idea for such a system was suggested to us by Deaf signers in Switzerland.
 

Overview of sign language translation pipelines

Machine Translation and Animation Assessment for Swiss German Sign Language

This two-part project dealt with Swiss German Sign Language. The first part involved sign language machine translation, the second sign language animation assessment, i.e., the evaluation of virtual signers, or 'avatars'.