The Color Imaging Pipeline for Digital Still and Video Cameras

IEEE ICIP Tutorial, San Diego, 2008

• Keigo Hirakawa, Harvard Unviersity
• Patrick Wolfe, Harvard Unviersity
• Truong Nguyen, University of California, San Diego

Tutorial Outline & Presentation Slides

• Overview, Color Processing
• overview of camera pipeline
• market trends, architecture, challenges for pipeline designs
• color processing inside camera
• Sampling, Artifacts, Noise Issues
• sampling, color filter array, interpolation
• crosstalk artifacts
• noise (Poisson vs Gaussian), interpolation+denoising
• Post-Pipeline: Enhancement, Display, Video
• image and video enhancement
• image super resolution
• frame rate conversion
• lens correction

Hot Topics & Outstanding Problems in Camera Pipeline

• Public perception of "megapixel count"
• Sensor noise problems--low light regimes
• Optical diffraction and minority carrier diffusion (crosstalk)
• Compressed dynamic range of CMOS sensors
• Sensor binning and artifacts
• Modularized ASIC/co-processor design
• Bad pixel amelioration
• Multiple illuminant estimation
• Image and color enhancement
• Storage/transmission of high data-rate multimedia
• Standards and their adaptation

Links, Database & Useful Codes

• Int'l Imaging Industry Association (I3A) Camera Phone Image Quality (CPIQ) Initiative
• DCRaw, open-source raw image conversion software, maintained by D. Coffin
• Collection of image denoising, coding, demosaicking methods, maintained by X. Li
• Collection of Poisson intensity estimation methods, maintained by P. Besbeas
• Collection of raw sensor images for white balance experiments, maintained by P. Gehler
• Collection of hyperspectral images of natural scenes, maintained by D. Foster

Resources

• IMA/MCIM industrial problems seminar on Algorithms for Digital Color Cameras by J.F. Hamilton (2006)
• CIS@RIT seminar on Spatio-Spectral Sampling & Crosstalk by Keigo Hirakawa (2008)
• IEEE SPS Santa Clara seminar on Demosaicking+Denoising & Spatio-Spectral Sampling by Keigo Hirakawa (2008)
• EI/CIC tutorial on System Interactions in Digital Color Imaging by R.Bala & G.Sharma (2009)

References

• "Single-Sensor Imaging: Methods and Applications for Digital Cameras," ed. R. Lukac, CRC Press, 2008.

Any Questions? Suggestions?