NEURAL IMAGE RE-EXPOSURE

ABSTRACT

The shutter strategy applied to the photo shooting process has significant influence on the quality of captured photograph. Improper shutter may lead to a blurry image, video discontinuity or rolling shutter artifact. Existing works try to provide an independent solution for
each issue. In this work we aim at re-exposing the captured photo in the post-processing, providing a more flexible way to address those issues within a unified framework. Specifically, we propose a neural network based image re-exposure framework. It consists of an encoder
for visual latent space construction, a re-exposure module for aggregating information to neural film with a desired shutter strategy, and a decoder for ‘developing’ neural film to a desired image. To compensate for information confusion and missing with frames, event stream, which could capture almost continuous brightness change, is leveraged in computing visual latent content. Both self attention layers and cross-attention layers are employed in the re-exposure module to promote interaction between neural film and visual latent content and information aggregation to neural film. The proposed unified image re-exposure framework is evaluated on several shutter-related image recovery tasks and performs favorably against independent state-of-the-art methods.

Source: Arxiv