Camera Noise with Normalizing Flows

A new model for camera noise based on normalizing flows

Released in: Modeling sRGB Camera Noise with Normalizing Flows

Contributor:

Summary

Noise modeling and reduction are fundamental tasks in low-level computer vision. They are particularly important for smartphone cameras relying on small sensors that exhibit visually noticeable noise. There has recently been renewed interest in using data-driven approaches to improve camera noise models via neural networks. These data-driven approaches target noise present in the raw-sensor image before it has been processed by the camera’s image signal processor (ISP). Modeling noise in the RAW-rgb domain is useful for improving and testing the in-camera denoising algorithm; however, there are situations where the camera’s ISP does not apply denoising or additional denoising is desired when the RAW-rgb domain image is no longer available. In such cases, the sensor noise propagates through the ISP to the final rendered image encoded in standard RGB (sRGB). The nonlinear steps on the ISP culminate in a significantly more complex noise distribution in the sRGB domain and existing raw-domain noise models are unable to capture the sRGB noise distribution. The authors propose a new sRGB-domain noise model based on normalizing flows that is capable of learning the complex noise distribution found in sRGB images under various ISO levels. Their normalizing flows-based approach outperforms other models by a large margin in noise modeling and synthesis tasks. They also show that image denoisers trained on noisy images synthesized with their noise model outperforms those trained with noise from baselines models.

2022

Year Released

Key Links & Stats

Modeling sRGB Camera Noise with Normalizing Flows

@InProceedings{Kousha_2022_CVPR, author = {Kousha, Shayan and Maleky, Ali and Brown, Michael S. and Brubaker, Marcus A.}, title = {Modeling sRGB Camera Noise With Normalizing Flows}, booktitle = {Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)}, month = {June}, year = {2022}, pages = {17463-17471} }

ML Tasks

  1. Denoising
  2. Image Generation

ML Platform

  1. Not Applicable

Modalities

  1. Still Image

Verticals

  1. General

CG Platform

  1. Not Applicable

Related organizations

Samsung AI Center–Toronto

York University

Vector Institute