Indian Sign Language recognition app — no gloves, no sensors, any phone

Every existing sign language recognition app either requires special gloves with sensors, a depth camera like Microsoft Kinect, or a machine learning setup that needs thousands of training images. None of them work on a normal phone. None of them are free. And almost none of them support Indian Sign Language specifically.

Voxetu's ISL recognition mode changes this. Open the page on any phone, hold your hand in front of the front camera, and sign a letter. The app recognises it and types it. No gloves, no depth sensor, no special hardware of any kind.

Who needs ISL recognition most

The obvious users are Deaf people who communicate in ISL. But there are three groups for whom this feature is particularly valuable in ways that are not immediately obvious:

Deaf-blind users — Someone who is both Deaf and visually impaired may know ISL but live with hearing carers who do not. When the user signs, the caregiver has no way to understand. Voxetu bridges this gap: the user signs, Voxetu speaks the word aloud, and the hearing carer understands. It acts as an interpreter without requiring an interpreter to be present.

ALS patients who grew up signing — A significant number of ALS patients are CODAs (children of Deaf adults) who learned ISL as a first or second language. As ALS progresses and voice is lost, these patients still have partial hand movement for much longer than they retain voice. ISL recognition lets them use a language they already know fluently, rather than learning a new AAC system from scratch.

ISL students and teachers — The live confidence percentage shown for each gesture provides instant feedback. A student signing the letter V at 91% confidence understands they are close to the correct shape. A student signing W at 43% knows immediately to spread their fingers more. This kind of real-time feedback is not available in any other free tool.

How it works — without the jargon

Your phone's front camera watches your hand continuously. An AI model identifies 21 specific points on your hand — fingertips, knuckles, wrist — 30 times per second. A classifier then reads the positions and angles of those points and determines which ISL letter shape they most closely match.

When the same letter is detected consistently for 1.5 seconds, it types automatically. The user builds up a word letter by letter, then presses Speak. The app speaks the completed word aloud.

All of this runs on the phone itself. No video is sent anywhere.

Three ways to sign

Mode	How it works	Best for
⏱ Hold 1.5 seconds	Hold the gesture steady — green timer fills — letter types	Most accurate, recommended for beginners
⟳ Continuous	Letter types immediately when detected	Fast, experienced signers
👆 Manual tap	Camera shows detected letter, user confirms by tapping	Maximum accuracy for important messages

Hindi output — speak what you sign

Voxetu includes a language toggle so signed messages can be spoken in Hindi rather than English. Common words like HELP, WATER, PAIN, YES, NO, DOCTOR, and MEDICINE are translated to Hindi automatically. For other words, phonetic transliteration is applied. This is particularly useful for Deaf users communicating with Hindi-speaking carers or family members.

The reverse direction — Speak to Sign

Voxetu also includes a speech-to-sign mode for the opposite situation: a hearing person speaks a word, and the app shows the ISL hand gesture for each letter, so a Deaf person can read the signs visually. This is useful when a hearing carer needs to communicate something specific and does not know ISL.

What the caregiver sees

Every signed message appears in the caregiver dashboard with a sign language icon, the full message text, and the time it was sent — exactly like any other communication mode. A Deaf patient's family can see everything communicated during the day in one place, even if they were not present.