A Neural Reflectance Field Model for Accurate Relighting in RTI Applications

What is it about?

Our paper, “A Neural Reflectance Field Model for Accurate Relighting in RTI Applications”, is finally out in ACM Transactions on Graphics! We present Neural Reflectance Fields (NRF), a novel approach for Reflectance Transformation Imaging (RTI) that enables high-quality, interactive relighting of objects under varying light directions. Models complex reflectance using implicit neural representations Captures fine surface details like shadows, highlights, and interreflections Achieves state-of-the-art visual fidelity with compact, lightweight models Introduces a new synthetic RTI dataset for benchmarking We also review related works, including Photometric Stereo, NeRF-based relighting, single-image relighting, and Bidirectional Texture Function (BTF) methods, highlighting their strengths and limitations for RTI applications.

Featured Image

Photo by Alexander Voronov on Unsplash

Photo by Alexander Voronov on Unsplash

Why is it important?

Example1 We propose a novel modeling approach for RTI that leverages recent advances in coordinate-based implicit neural representations (INRs) for discrete low-dimensional signals. We model T as an MLP-based INR trained directly from MLIC samples, by incorporating positional encoding shown to be critical for capturing high-frequency details. Furthermore, unlike most INR methods that rely solely on input coordinates, we introduce a compact per-pixel latent vector that encodes local reflectance variations. These latent vectors are learned via a CNN jointly trained with the INR and stored alongside the network weights. This design enables accurate reproduction of both global illumination effects (e.g., self-shadows, interreflections) and fine local details. Example2 To evaluate our method, we present a novel synthetic RTI dataset generated. Existing synthetic datasets often suffer from limitations: small MLIC sizes, absence of complex shadows,s and specularities, and incomplete coverage of the incident lighting hemisphere , hindering generalization to new illumination conditions. Our dataset addresses these issues by providing large-scale MLICs captured in a virtual dome-light configuration and including objects with intricate geometries and material properties. Example2 Our method uses a lightweight MLP network, which can relight images in real time.

Perspectives