Stereo Vision with Water Surface Estimation and Refraction-Correction for Robot Manipulation of Underwater Objects

Christian Deus Telmo Cayao (1351204)

Stereo vision uses a pair of cameras similar to humans for depth perception and distance estimation. Stereo vision is commonly used as a sensing unit for robots due to the simplicity of its hardware. However, traditional implementation of stereo algorithms do not work well when considering scenes under the effects of refraction (e.g., looking at an object submerged in water from the air). Thus, to allow a robot to sense and manipulate an underwater object, there is a need to modify the stereo vision algorithms to adapt to situations under the effects of refraction. With that purpose, we propose approaches to compensate for the error caused by refraction in depth estimation in the case where we are looking at an underwater object from an aerial perspective. It is necessary to determine the geometric configuration of the refractive interface to correct the effects of refraction. This thesis presents methods to estimate the geometric configuration of the water surface. We estimate the water surface configuration first by using a fiducial marker and next by using matching features from the stereo images.

Furthermore, we consider two major applications of refraction-correction. These applications include stereo reconstruction and object segmentation and tracking. First, the proposed methods to estimate the refractive interface together with refractive stereo matching are used to generate a refraction-corrected disparity map which is used to reconstruct the scene in 3-D. Next, the proposed refractive triangulation is used to estimate the refraction-corrected distance of an underwater target using matching image features. The method can be implemented in real-time and is designed to work in conjunction with object segmentation algorithms for real-time object tracking since we would like to use the stereo vision system to estimate the location of an underwater target for robot manipulation.

A quantitative evaluation of the overall reconstruction accuracy is performed for the generated 3-D reconstructions. Evaluation results show that the use of such method effectively compensates for refraction with accuracy comparable to that of the stereo block matching implementation in air (without refraction). The accuracy of the distance estimation using refractive triangulation is also evaluated and the results show that the method is able to reduce the error to tolerable ranges at different water depths. This results prove the potential of using the developed stereo vision system for robot manipulation.