Machine Learning-Based Monte Carlo Denoising

Tech ID: 32590 / UC Case 2015-002-0

Background

Monte Carlo (MC) rendering algorithms, such as path-tracing, can produce highly realistic images of virtual scenes. However, the process requires long rendering times (often on the order of days), making it unusable for real-time applications and unwieldy even for offline, high-end feature-film production.  The rendering times can be reduced by computing fewer rays, but this results in objectionable noise in the final image. MC noise has proven to be a difficult obstacle to overcome, and there is a need for fast techniques that eliminate it for images with arbitrary distributed MC effects (such as depth-of-filed, motion blur, area shadows, and global illumination) while preserving image quality.

Description

In 2015, researchers at the University of California, Santa Barbara applied machine learning (ML) to the task of MC noise reduction by outputting basic scene features from the rendering system (such as surface normal, texture information, depth, position, etc.) and feeding them into a learning model. The model is then trained to remove MC noise and produce high-quality images. The architecture of the learning system is flexible, and could involve MLPs, CNNs, transformers, deep networks, or other machine learning models, and the features input from the rendering system could vary as well. The system can be trained in real time during rendering or offline on an existing dataset with few-sample renderings coupled with ground truth. ML-based MC denoising overcomes the “last mile” problem of slow convergence that is fundamental to other approaches and produces high-quality images orders of magnitude faster than previously possible. 

Advantages

  • Fast filtering of rendered images, producing high-quality results orders of magnitude faster than with other techniques
  • Avoids costly computation process of additional sample light rays required by conventional MC renderers
  • Works on a wide range of distributed MC effects such as depth-of-field, motion blur, area lighting, glossy reflections, and global illumination
  • Simple to integrate into existing Monte Carlo rendering systems

Applications

  • Graphics rendering
    • Gaming
    • Film production
    • Architecture
    • Marketing/Advertising

Patent Status

Country Type Number Dated Case
United States Of America Issued Patent 10,832,091 11/10/2020 2015-002
United States Of America Issued Patent 10,192,146 01/29/2019 2015-002
 

Related Materials

Contact

Learn About UC TechAlerts - Save Searches and receive new technology matches

Inventors

Other Information

Keywords

monte carlo, MC, machine learning, image, noise, rendering, render, film, gaming, marketing, advertising, Architecture

Categorized As