3D reconstruction and rendering for remote sensing images

Objective

Realistic 3D reconstruction from multiple Remote Sensing (RS) images has been studied as a Multi-View Stereo process, in which 3D scenes are reconstructed progressively in terms of geometry and appearance. This method can introduce artifacts and compromise realism. Recently, a new framework in neural rendering, the Neural Radiance Field (NeRF) and its variants, e.g. 3D-GS, which model both geometry and appearance of a scene within a radiance field, has delivered a more immersive scene. However, as with any burgeoning technology, NeRF has encountered its share of challenges and limitations, particularly in terms of computational efficiency and incapacity of modeling many ubiquitous, real-world phenomena in uncontrolled images. 3D-RTR aims at building a new 3D reconstruction framework to address these issues and provide high-quality, immersive experience of 3D city models. To build it, a unified, explicit representation of the complete imaging chain, which models four key physical entities (i.e. the light source, the atmosphere, the scene and the camera), is proposed to describe the real-world phenomena in a more direct way. This unified imaging chain is further transformed into a differentiable, highly parallelized architecture designed for learning realistic 3D scenes from remote sensing images, with various optimization tactics employed to accelerate the learning process. 3D-RTR will be tested using remote sensing data acquired on various locations and make these reconstructed models accessible to the public online. To ensure the successful implementation of these project goals I will conduct my research at the 3DOM research unit of FBK Trento, with the guidance of Prof. Fabio Remondino, a leading scholar. This project will offer new solution for smart city applications and digital twin generations, while also preparing me to become a tenured scholar and obtain advanced funding, strengthening my academic profile and allowing me to establish my own research group.