Irssa — Pakistan Sign Language
Offline Android learning game for deaf children — 37 Urdu PSL letters, 26 English letters, and an on-device MediaPipe hand coach.
Irssa is a Kotlin and Jetpack Compose app that teaches Pakistan Sign Language to deaf and speech-impaired children. It keeps the two alphabets deliberately separate — 37 Urdu PSL letters taught with real hand photographs and Urdu words a child already knows, and 26 English letters taught as a distinct language — so the two sign systems are never confused. An offline recognition coach uses Google MediaPipe Hand Landmarker to find 21 hand landmarks and a nearest-neighbour classifier trained on real hands from independent signers. Nothing is uploaded: camera frames are processed on the device and discarded. The app ships as a downloadable APK and the repository includes the capture and training scripts so a child can teach the recognizer their own hand.
Key Features
- 37 Pakistan Sign Language letters and 26 English letters, taught as separate alphabets
- Fully offline camera coaching for 37 PSL handshapes and 24 static English letters
- MediaPipe Hand Landmarker plus a nearest-neighbour classifier trained on independent signers
- Strict acceptance gate tuned on held-out data — it prefers "not yet" over a false success
- Numbers 0–50, everyday words, quizzes, camera quests, stickers and a gesture converter
- Personal hand capture and hybrid retraining tools so the model learns one specific hand
- No accounts, no ads, no analytics — frames never leave the device
Repository Topics
Challenges
Recognising the same handshape across different signers without producing false successes, and separating genuinely static letters from motion-dependent signs that a single-frame classifier cannot honestly grade.
Results
A ready-to-install Android APK plus a reproducible capture-and-train pipeline, with English J and Z left as guided practice rather than faked recognition.