Optical Flow-based Temporal Video Analysis for Hand Hygiene Monitoring

Abstract

Touching people or surfaces can increase spread of infectious diseases. Successful hand sanitization performed with correct sequences of hand movements as described by WHO is an important step in maintaining hygiene and decreasing contamination. In this work, deep learning models are trained with inputs enriched with optical flow vectors for the recognition of hand hygiene movements differing from prior approaches. Optical flow describes the apparent movement of pixels in the image. Optical flow vectors are used as extra channels alongside the original image additionally masking them with the hand region of interest or applying image preprocessing techniques. These multi-channel inputs are used to train three different deep learning models. Experimental results show that optical flow improves performance when used along with the original images. The best results are obtained when CNN models are trained on multi-frame images including the angle of optical flow.