Lightweight Gradient-Aware Upscaling of 3D Gaussian Splatting Images

ICCV 2025

Simon Niedermayr, Christoph Neuhauser, Rüdiger Westermann
Technical University of Munich
arXiv 2D Demo 3D Demo Code

TLDR: We utilize analytical image gradients to upscale 3DGS renders to improve rendering and reconstruction speed.

Abstract

We introduce an image upscaling technique tailored for 3D Gaussian Splatting (3DGS) on lightweight GPUs. Compared to 3DGS, it achieves significantly higher rendering speeds and reduces artifacts commonly observed in 3DGS reconstructions. Our technique upscales low-resolution 3DGS renderings with a marginal increase in cost by directly leveraging the analytical image gradients of Gaussians for gradient-based bicubic spline interpolation. The technique is agnostic to the specific 3DGS implementation, achieving novel view synthesis at rates 3x-4x higher than the baseline implementation. Through extensive experiments on multiple datasets, we showcase the performance improvements and high reconstruction fidelity attainable with gradient-aware upscaling of 3DGS images. We further demonstrate the integration of gradient-aware upscaling into the gradient-based optimization of a 3DGS model and analyze its effects on reconstruction quality and performance.

Motivational Example

Traditional bicubic interpolation often inaccurately reconstructs signals due to its reliance on finite difference approximations of gradients, leading to artifacts. Enhancing interpolation by incorporating analytical gradients at each sample point allows for a more precise spline fit. This approach leverages the exact rate of change of the signal, resulting in a reconstruction that closely mirrors the original, effectively capturing subtle variations and reducing interpolation errors.​


Finite Differences

Analytical Gradients

Method

Bicubic interpolation / upscaling fits a bicubic polynomial through a set of neighboring pixels using finite differences to estimate the color gradients. We replace this gradient estimation with the real analytical gradients of the 2D Gaussians. The exact image gradients allow for a more accurate interpolation and therefore image upscaling.

Since the upscaling operation is fully differentiable we can include the upscaling in the reconstruction pipeline and therefore reducing the computational load of the rasterization step and the whole reconstruction process.


Gradients are used to upscale the image with spline interpolation to the target resolution.

A 2D Gaussian Splatting model is fitted to the ground truth image, and upscaling methods are applied to the image rendered at ¼ the resolution of the original.

Interactive Web Demo for 2D Images

This interactive demo shows our upscaling method (and others) applied to 2D images. The image is a Gaussian Mixture Model that was reconstructed with an approach similar to GaussianImage. The images are rendered at a lower resolution (Upscale Factor) and then upscaled with the selected method to match the screen resolution.
Note: The improvement over bicubic upscaling for 2D images is not as great as for 3DGS, since the number of Gaussians contributing to each pixel is much lower.

Explore the Demo for the entire DIV2K and Kodak dataset

Web Demo

This is an interactive demo showcasing our 3D Gaussian Splatting Upscaling method in a web browser using WebGPU. The demo was built ontop of brush. Source code can be found here.

Your browser does not support WebGPU. Please use a compatible browser.

Video Results

Video results for 8x upscaling with different methods. NinaSR1 is a Deep Learning upscaler from TorchSR.

Note: View the videos in full screen to see the differences between the methods.

BibTeX

@misc{niedermayr2025upscaling3dgs,
    title     = {Lightweight Gradient-Aware Upscaling of 3D Gaussian Splatting Images}, 
    author    = {Simon Niedermayr and Christoph Neuhauser Rüdiger Westermann},
    year      = {2025},
    booktitle = {Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)},
    year      = {2025},
}