arXiv 2023 DatasetsSign LanguageCrowdsourcing

Bornil: An Open-Source Sign Language Data Crowdsourcing Platform for AI Enabled Dialect-Agnostic Communication

Shahriar Elahi Dhruvo, Mohammad Akhlaqur Rahman, Manash Kumar Mandal, Md. Istiak Hossain Shihab, A. A. Noman Ansary, Kaneez Fatema Shithi, Sanjida Khanom, Rabeya Akter, Safaeid Hossain Arib, M.N. Ansary, Sazia Mehnaz, Rezwana Sultana, Sejuti Rahman, Sayma Sultana Chowdhury, Sabbir Ahmed Chowdhury, Farig Sadeque, Asif Sushmit

arXiv preprint arXiv:2308.15402, 2023

Paper arXiv BibTeX

TL;DR

Sign language AI is bottlenecked by data. Bornil lets signers record, annotators label at sentence and gloss level, and validators check quality, all in one crowdsourcing platform.

  • 73 hof Bangla Sign Language video in BornilDB v1.0
  • 21,154recorded samples
  • 25,572unique Bengali words (138,586 total)
Overview figure for Bornil: An Open-Source Sign Language Data Crowdsourcing Platform for AI Enabled Dialect-Agnostic Communication
The Bornil platform. Validators review recordings and metadata (top); annotators align sentence and gloss labels to the video timeline (bottom).

Abstract

The absence of annotated sign language datasets has hindered the development of sign language recognition and translation technologies. In this paper, we introduce Bornil; a crowdsource-friendly, multilingual sign language data collection, annotation, and validation platform. Bornil allows users to record sign language gestures and lets annotators perform sentence and gloss-level annotation. It also allows validators to make sure of the quality of both the recorded videos and the annotations through manual validation to develop high-quality datasets for deep learning-based Automatic Sign Language Recognition. To demonstrate the system's efficacy; we collected the largest sign language dataset for Bangladeshi Sign Language dialect, perform deep learning based Sign Language Recognition modeling, and report the benchmark performance. The Bornil platform, BornilDB v1.0 Dataset, and the codebases are available.

Platform

  1. RecordSigners record sentences shown on screen, with metadata such as lighting, camera distance, and viewpoint.
  2. AnnotateAnnotators add sentence-level and gloss-level labels aligned to the video timeline.
  3. ValidateValidators check both the recordings and the annotations, keeping dataset quality high.

Citation

@article{dhruvo2023bornil,
  title   = {Bornil: An Open-Source Sign Language Data Crowdsourcing Platform for
             {AI} Enabled Dialect-Agnostic Communication},
  author  = {Dhruvo, Shahriar Elahi and Rahman, Mohammad Akhlaqur and Mandal, Manash Kumar and
             Shihab, Md. Istiak Hossain and Ansary, A. A. Noman and Shithi, Kaneez Fatema and
             Khanom, Sanjida and Akter, Rabeya and Arib, Safaeid Hossain and Ansary, M. N. and
             Mehnaz, Sazia and Sultana, Rezwana and Rahman, Sejuti and Chowdhury, Sayma Sultana and
             Chowdhury, Sabbir Ahmed and Sadeque, Farig and Sushmit, Asif},
  journal = {arXiv preprint arXiv:2308.15402},
  year    = {2023}
}