Project 57: NeRF 3D Rendering for Autonomous Driving Scenes

Contact Information:

Prof. Wang Hesheng

Project Description and Objectives:

Accurate perception of the environment in autonomous driving is crucial for achieving successful autonomous driving. However, deep perception algorithms require data from various scenarios, some of which are difficult to collect, such as unconventional scenes like traffic accident sites. The data distribution in such cases is severely imbalanced, leading to poor performance of deep neural networks in extremely rare scenarios. To address the challenges of data collection and data imbalance, we aim to build an architecture for autonomous driving scene modeling and synthesis of anomalous data based on neural radiance fields. We aim to reconstruct three-dimensional autonomous driving scenes from existing data using implicit representations. Additionally, we will research how to edit well-modeled scenes and generate more tail data through scene object editing and new viewpoint rendering.

Eligibility Requirements:

Fluent English writing and speaking skills are required.

Basic knowledge of deep learning.

Proficient programming skills in Python.

Main Tasks:

Develop prototype software.

Complete internship reports.

Conduct two research presentation sessions (a. Literature review; b. Technical demonstration).

Submit a paper to a journal.

Website:

Lab: https://irmv.sjtu.edu.cn/

School: http://english.seiee.sjtu.edu.cn/