American Sign Language is the most predominant sign language for deaf people in the US. Yet, very few amount of people from the population actually know ASL. This app lets everyone learn ASL in a very easy way and fun environment!

Team Members

Launch Site

Coach Me ASL

Learn ASL really easily!

V800 team b4cc5fe80b4660b5bac576eb2ebb128f 1537198858

Description

CoachMeASL

A ReactJS app that lets you learn the American Sign Language with the power of Machine Learning.

The frontend of the app is taken care by ReactJS and the backend is taken care by TensorFlow.js This app has a chatbot interface that lets users type, ask questions and play the game. The chatbot gives the user a sign that he has to make and the Machine Learning model will recognize and classify whether the sign made by the user was correct or not. The user has a points score which tells the user the number of correct signs he/she got from the total number of signs they tried.

Currently, the model only recognizes 5 signs, 2 letters namely A and Y and 3 words namely PEACE, OKAY and THUMBS UP.

Letters

A Y

Words

PEACE OKAY THUMBS UP

Learning ASL can get hard for a lot of people. The flexibility on its own could be a problem for many. The main purpose of this app is to let users learn ASL with practice and instead of an actual human telling whether the user is right or wrong, processing the information through a bot which is much more efficient and much more easier for the users themselves. The app is portable and hence it can run on all devices since it uses TensorFlow.js

Demo

Demo Click here to check out the live version.

Instructions

Using the app is pretty straightforward, the steps involved are given below: 1. Click the guess button. 2. Once the popup opens, click the Check Answer button and wait. 3. You will now be able to see live video feed, now just make any sign from these 5 mentioned below, i. Peace ii. Okay iii. A iv. Y v. Thumbs Up 4. On the console, you can see whether the sign you made was correct or not.

View the screencast:
https://media.giphy.com/media/4Z5mNlSFzBnX0qQonp/giphy.gif

Built With

Resources

The resources used in the app are mentioned below: * IBM Watson * TensorFlow.js * p5.js * create-react-app * React component: Circular progress bar, Sidebar, Popup

Feedback / Ratings (5)

All Feedback