Computer Vision Solution for Handwashing Quality Scoring
- #AI
- #Audio/video analytics
- #Computer vision
- #Data science
- #Healthcare
- #Machine learning
About the Client
A client is a medical service provider who is innovating the handwashing experience.
Business Challenge
Our client has created a unique technological solution for hand hygiene. This solution controls hand washing by scanning the hands’ movements and providing immediate feedback on the quality of the wash cycle, thus ensuring thorough hand hygiene.
Since the client had limited time to take the product to the market, they have decided to partner with Quantum to do some data science work.
Solution Overview
The goal of the project for Quantum was to detect handwashing gesture recognition and deploy it to the washing station to provide users with real-time results on the quality of their actions.
The final goal of the client is to create a platform that monitors, coaches, learns, and utilizes data science to provide visual validation reports on the health and quality of the wash. The ML model for gesture recognition developed by Quantum has allowed the client to achieve 80% accuracy, and go into production in less than 2-months time.
Project Description
In order to achieve the best possible results the project was split into a couple of stages:
Data Labeling and preparation
For better results, we used videos from the target platform. The dataset was labeled according to possible hand wash steps. Assuming the steps are non-overlapping, each step was labeled according to its starting and ending time in the video. Assuming the effort needed to get videos is significant, we use around 100 videos for the baseline development with a train/test split of 60-40.
Modeling
We built a handwashing event detector. The events are different stages in the handwashing process. Our system outputs timestamps and event classification so our client’s systems are able to compare a handwashing procedure with the World Health Organization’s recommended procedure.
At the core of the event detector is a neural network, which utilizes both spatial and temporal information to give accurate event classification for each timestep. The predictions are further filtered based on the distribution of the handwashing event times for each particular event class.
Deployment to the hardware platform
The developed algorithm is deployed in two different environments. An AWS g3s.xlarge instance is used for on-demand cloud processing. A Raspberry Pi 3b with an Intel Compute Stick 2 module is used to deliver real-time processing on the hand hygiene station itself. RGB video from the Pi Camera v2 is used in the hygiene station.
Let's discuss your idea!
Technological Details
Both the model and the infrastructure are written in Python. During the development of the handwashing pose classification the following algorithms were used:
- PyTorch for modeling with neural networks;
- OpenCV to work with video;
- OpenVINO for communicating with the Intel Compute Stick;
- NumPy to work with matrices;
- Pandas to work with tabular data;
- Sklearn for data science utilities.
The system is deployed to AWS for cloud computing and to a Raspberry Pi 3b with an Intel Compute Stick 2 for local processing.
