High Dynamic Range (HDR) Digital Imaging with Neural Networks

Background

Standard digital cameras typically take images with under/overexposed regions because of their sensors’ limited dynamic range. The most common way to capture high dynamic range (HDR) images using these cameras is to take a series of low dynamic range (LDR) images at different exposures and then merge them into an HDR image. Producing a high dynamic range (HDR) image from a set of images with different exposures is a challenging process for dynamic scenes. A category of existing techniques first register the input images to a reference image and then merge the aligned images into an HDR image. However, the artifacts of the registration usually appear as ghosting and tearing in the final HDR images.

Technology Description

Researchers at UC San Diego have described the first learning-based technique to produce a high-dynamic-range (HDR) digital image using a set of low-dynamic-range (LDR) images captured from a dynamic scene. A convolutional neural network is used to generate the HDR image from a set of images aligned with optical flow. To properly train the network, a strategy is given to produce a set of input LDR images, with varied exposure, and their corresponding ground truth image. Three architectures for the learning-based techniques are examined and demonstrate that this model leads to improvement in dynamic settings where varied exposure and movement must be addressed.

Applications

This technology will find ready application in digital photography, surveillance, autonomous systems and other image capture applications

Advantages

The algorithm takes a set of low dynamic range (LDR) images of DYNAMIC SCENES with different exposures as the input and outputs a high dynamic range (HDR) image. Alternatively, our method can take a set of noisy images of dynamic scenes with the same short exposures and generate a clean (denoised) image that are of a high quality.

State Of Development

We demonstrate the performance of our system by producing high-quality HDR images from a set of three LDR images. Experimental results show that our method consistently produces better results than several state-of-the-art approaches on challenging scenes

Intellectual Property Info

A provisional patent has been submitted and the technology is available for licensing.

Related Materials

• Nima Khademi Kalantari and Ravi Ramamoorthi. 2017. Deep High Dynamic Range Imaging of Dynamic Scenes. ACM Trans. Graph. 36, 4, Article 144 (July 2017), 12 pages. - 07/01/2017

Patent Status