A Machine Learning System For Human-in-the-loop Video Surveillance

Abstract

We propose a novel human-in-the-loop surveillance system that continuously learns the properties of objects that are interesting for a human operator. The interesting objects are automatically learned by tracking the eye gaze positions of the operator while he or she monitors the surveillance video. The system automatically detects interesting objects in the surveillance video and forms a new synthetic video that contains interesting objects at earlier positions in the time dimension. The operator always views this synthetically formed video which makes manual video retrieval tasks more convenient. Sensitivity to operator interests and interest changes are other major advantages. We tested our system both on synthetic and real videos, which are provided as supplementary materials [1]. The results show the effectiveness of the proposed system.