Noise reduction in Foveon Images.

Noise is a constant problem in Digital Photography. Resulting from the random effects of heat on the light sensitive diode junctions in the camera's sensor chip, it has an effect similar to that of grain in conventional silver based photography.

Noise is most noticeable in large areas of plain colour, such as a clear blue sky, or in a plain background to a portrait, and in many cases can completely ruin the intended effect of an image.

The Foveon chip, as everyone must know by now, is a sensor chip for digital cameras which has an unusual arrangement with sensor cells placed one above the other in a stack formation.

One of the major criticisms which has been aimed at the Foveon sensor is that it is is noisy, compared to similar sized conventional Bayer grid sensors. This is certainly true although experience has shown that it is still 'quieter' than the smaller chips used in most compact domestic digital-cameras.

Having a Sigma SD-9 and being disappointed by a set of pictures which had turned out to be rather noisy, I decided to investigate the noise further and see if there was any possible solution to the noise problem.

Comparing the images to those from my other digital cameras, I realised that the level of noise on the 'noisy' images was in fact no greater than that on images from the other cameras that appeared 'quiet'. The basic difference however was that the noise was of a far more visible nature, the reason being that the maximum noise occurs in the Green channel.

Our eyes are most attuned to detail in the Green channel, which is why Bayer grid chips have twice as many Green sensors as either of the other colours. We are therefore very sensitive to any noise which may disrupt the detail in Green.

With a Bayer sensor chip, twice as much area of the surface of the sensor is devoted to producing the Green channel. As a result the Green channel is relatively quiet whilst the Red and Blue channels can be quite noisy without the noise becoming obvious.

With the Foveon chip however, the noise in the Green channel is very noticeable, and results in most of the noise visible in the image. However the unique structure of the Foveon chip also gives us a way of eliminating much of this noise.

The Foveon chip has three layers of photosensitive semiconductor junctions. The lowest layer is sensitive to Red light, the next layer being sensitive to both Red and Green Light. The final, top layer is sensitive to Red, Green and Blue light. In order to measure the level of Green light, it is necessary to subtract the output of the Red junction from the output of the Green junction. Similarly the output of the Blue junction has to have the output of the Green junction subtracted from it. This has interesting repercussions for the thermal noise generated in the junctions.

As a result of the above subtraction process, the Green output contains the noise from the Green junction, but also contains the noise from the Red Junction. The Blue output from the chip also contains the noise from the Green junction. This is a totally different situation to that on a conventional Bayer grid chip, where the noise output of each set of sensors is completely independent of the others.

Knowing the above, it is fairly easy, with a little bit of arithmetic, to produce an image of the noise in the Green channel, with a small amount of spurious colour information, but with virtually no luminance information present.

Original Image	Extracted Noise (enhanced for clarity)

By then treating the resulting image with a high-pass spacial filter, the majority of the colour information can also be removed, leaving just a copy of the noise in the channel. Once you have this noise, it is a reasonably simple matter to remove it from the final image, thus considerably reducing the final noise in the completed picture.

Original Image	Processed Image

In order to carry out this process out I have written a script for Paint Shop Pro version 8 and an action for Photoshop CS, which carry out the necessary calculations. They do not work in precisely the manner described above, but essentially use the same principle. In practice the PSP script also requires some extra processing, as carrying out the mathematics in just 8 bits, cannot be done without numeric overrun. This causes major artifacts which have to be removed.
This is not necessary with the Photoshop action as Photoshop calculates to an accuracy of 16 bits.

To use the script, first load it into your 'trusted scripts' directory. Load the noisy image into PSP, then run the script. When the script has completed, the image will have another layer added which contains the noise reduced image. By putting the new image onto a separate layer, it is easy to see the noise reduction by switching the layer's visibility on and off. This also makes it possible to see any resulting artifacts, and remove them by adding a mask through to the original image.

Possible artifacts to look out for are:-

Softening of part of the image - Inevitably some of the image may emulate the type of noise which is being removed. This causes that part of the image to lose detail. The effect is normally fairly slight and will often only be noticed when comparing the original and processed images.

Original Image	Processed Image

Loss of colour - Some small parts of the image may lose colour saturation. Again this may only be noticed when comparing the processed image with the original.

Original Image	Processed Image

Fringing - some parts of the image may acquire a fringe, not dissimilar to the purple fringes which occur with Bayer grid sensors. This tends to occur with items that are blue or pink, or which are against a blue or pink background.

The script.

The script is called Remove Foveon Noise.PspScript.

The script will not work correctly if:-

The image has been resized or resampled.

The image has previously been processed by another noise removal algorithm.

The colour resolution of the image has been reduced, as happens with most JPeg compression algorithms (including that in the original version of Sigma Photo Pro).

The image was produced from any source other than a Foveon imaging chip.

The script only removes noise from the Green channel of the image.

Why only the Green channel?

Noise is most visible in the Green channel but exists in all three colour channels of the image, however the noise in the Red channel cannot be removed by this method. Blue channel noise can in theory be removed, but in practice, the result is more artifacts from the processing in return for very little reduction in visible noise.