Machine Learning-Based Monte Carlo Denoising

Monte Carlo (MC) rendering algorithms, such as path-tracing, can produce highly realistic images of virtual scenes. However, the process requires long rendering times (often on the order of days), making it unusable for real-time applications and unwieldy even for offline, high-end feature-film production.  The rendering times can be reduced by computing fewer rays, but this results in objectionable noise in the final image. MC noise has proven to be a difficult obstacle to overcome, and there is a need for fast techniques that eliminate it for images with arbitrary distributed MC effects (such as depth-of-filed, motion blur, area shadows, and global illumination) while preserving image quality.

In 2015, researchers at the University of California, Santa Barbara applied machine learning (ML) to the task of MC noise reduction by outputting basic scene features from the rendering system (such as surface normal, texture information, depth, position, etc.) and feeding them into a learning model. The model is then trained to remove MC noise and produce high-quality images. The architecture of the learning system is flexible, and could involve MLPs, CNNs, transformers, deep networks, or other machine learning models, and the features input from the rendering system could vary as well. The system can be trained in real time during rendering or offline on an existing dataset with few-sample renderings coupled with ground truth. ML-based MC denoising overcomes the “last mile” problem of slow convergence that is fundamental to other approaches and produces high-quality images orders of magnitude faster than previously possible.