Balancing Incident and Ambient Light for Illumination Compensation in Video Applications

Hand-gesture interaction with a smart TV using a conventional camera under restricted ambient illumination is complicated by the varying incident illumination coming from the screen. In this paper, we propose a method for compensating for the poor ambient illumination in the scene captured by a smart TV camera by balancing it against this kind of incident illumination. The proposed framework models the variations in the ambient illumination as a factor of the varying incident illumination caused by the changing multimedia content displayed on the smart TV. Further, by estimating the average incident illumination and measuring its deviation during the scene transition, an illumination intensity correction factor is deduced, and thus ambient illumination is corrected iteratively. To validate the effectiveness of the proposed method, we benchmark it against multiple video sequences depicting the human gesture interaction with a smart device under both low-lighting (typically in the night) and natural-lighting conditions. Furthermore, experiments on such challenging video sequences have demonstrated improved accuracy and robustness of visual target trackers by preprocessing the sequence using our method.