GestureCom: Bridging the communication gap

Explore your world of sign with us!

GestureCom is a privacy first, offline capable sign language transcriber.
It comes with a pre-build model to get started and also offers the user, the ability to train their own model
All the data is processed on the device and it never leaves the device.

Enjoy the privacy focussed, offline (partial) experience on your own by checking the following features

1
Start detecting sign languageUse or pre-trained model or try your custom model
2
Start training your custom modelCollect and train your own custom model.

Default Model:

The default model can recognise 7 general signs.
The gestues that are recognised by the default model is:

  • yes
  • no
  • hello
  • peace
  • i love you
  • thank you
  • please

Check it out by yourself here:

Use cases:

  • Facilitate communication with differently abled people on the fly.
  • Used as a communication between children and elders.
  • Used in a public place to facilitate the communication between common people and differently abled people.
  • Used by emergency services to understand the situation of a differently abled person.
  • Many more...

Technologies used:

Norway

Tensorlfow (Python)

Machine Learning

Used tensorflow-python to learn and train the initial 2 models

Norway

Tensorlfow (Java Script)

Machine Learning

Used tensorflow-js to facilitate the training and running of both pre-built and custom model right in your device

Norway

Mediapipe (Web)

Landmark Detection

Used mediapie to detect both pose and handlandmark. Thiseliminates the requirement to store or process personally identifiable information by converting the images to points in the space.

Norway

HTML

Web

Used to make the skeleton of the site.

Norway

Javascript

Web

Javascript is used to run and handle the used interaction in the web. This also facilitate the intervace with webGl.

Norway

CSS

Web

Css is used to style the site and make it look prety

Norway

WebGl

Web|Computing

WebGl is used to run the models in the machine directly. This eliminates the slow VM of JS and can handle large datasets with ease

Norway

Git

Version Control

Git is used to manage the different code versions and allowed to easily find and rectify the mistakes. It also acted as backup to safeguare the source code.

Norway

Python

Computing

Python is used to explore the initial idea and train the first few models.

Norway

NextJS

Web

NextJS is used to streamline the developed experience and facilitate faster iteration

Norway

Github

Version Control|Publishing

Github is used as remote code storace

Want to see the source code? See here

flowChartDetectionflowChartModelTraining

Future Plans:

  • Add notification support to send notification upon recognising a particular sign
  • Expand the default model to recognise more general symbols