Computer Graphics Forum (Proc. of Pacific Graphics) 2023
Chongrui Fan ¹ ² Yiming Lin ² Abhijeet Ghosh ¹ ²
Imperial College London ¹
Lumirithmic ²
Patent pending
Fig. 1: The estimated Bidirectional Reflectance Distribution Function (BRDF) and depth maps(c – g) of a wooden elephant are obtained from two input images (a, b) captured using a smartphone with a multi-lens imaging system. A probe is provided in the corners of input images to indicate the lighting conditions during the capture (not being used as an input). This enables realistic relighting rendering (h) under the Grace Cathedral lighting environment.
Abstract:
We present a deep neural network-based method that acquires high-quality shape and spatially varying reflectance of 3D objects using smartphone multi-lens imaging. Our method acquires two images simultaneously using a zoom lens and a wide-angle lens of a smartphone under either natural illumination or phone flash conditions, effectively functioning like a single-shot method. Unlike traditional multi-view stereo methods which require sufficient differences in viewpoint and only estimate depth at a certain coarse scale, our method estimates fine-scale depth by utilising an optical-flow field extracted from subtle baseline and perspective due to different optics in the two images captured simultaneously. We further guide the SVBRDF estimation using the estimated depth, resulting in superior results compared to existing single-shot methods.
Publication: Deep Shape and SVBRDF Estimation using Smartphone Multi-lens Imaging. Chongrui Fan, Yiming Lin, and Abhijeet Ghosh. Computer Graphics Forum (Proc. Pacific Graphics), 42(7), 2023.
