VRS-NeRF: Accelerating Neural Radiance Field Rendering with Variable Rate Shading

Recent advancements in Neural Radiance Fields (NeRF) provide enormous potential for a wide range of Mixed Reality (MR) applications. However, the applicability of NeRF to real-time MR systems is still largely limited by the rendering performance of NeRF. In this paper, we present a novel approach fo...

Full description

Saved in:
Bibliographic Details
Main Authors: Rolff, Tim, Schmidt, Susanne, Li, Ke, Steinicke, Frank, Frintrop, Simone
Format: Conference Proceeding
Language:English
Subjects:
Computer graphics
Computing methodologies
Graphics systems and interfaces
HCI design and evaluation methods
Human computer interaction (HCI)
Human-centered computing
Image edge detection
Measurement
Merging
Mixed reality
Neural networks
neural radiance fields
psychophysical experiment
Training
User studies
variable rate shading
virtual reality
Visualization
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Recent advancements in Neural Radiance Fields (NeRF) provide enormous potential for a wide range of Mixed Reality (MR) applications. However, the applicability of NeRF to real-time MR systems is still largely limited by the rendering performance of NeRF. In this paper, we present a novel approach for Variable Rate Shading for Neural Radiance Fields (VRS-NeRF). In contrast to previous techniques, our approach does not require training multiple neural networks or re-training of already existing ones, but instead utilizes the raytracing properties of NeRF. This is achieved by merging rays depending on a variable shading rate, which reduces the overall number of queries to the neural network. We demonstrate the generalizability of our approach by implementing three alternative functions for the determination of the shading rate. The first method uses the gaze of users to effectively implement a foveated rendering technique in NeRF. For the other two techniques, we utilize shading rates based on edges and saliency. Based on a psychophysical experiment and multiple image-based metrics, we suggest a set of parameters for each technique, yielding an optimal tradeoff between rendering performance gain and perceived visual quality.
ISSN:2473-0726
DOI:10.1109/ISMAR59233.2023.00039