Demosaicking Algorithm for the X-Trans Color Filter Array

In my last post I showed that ACDSee´s demosaicking algorithm for X-Trans images has a few issues. Well, interpolating the X-Trans color filter array (CFA) is not as easy as interpolating a Bayer CFA, simply because the X-Trans CFA is much more complex. However, Fuji (inventor of the X-Trans CFA) claims that their layout can outperform the Bayer CFA in terms of image quality. Obviously this is only true when applying a demosaicking algorithm which is sophisticated enough to really handle the CFA.
Some time ago in 2013 I wrote my own demosaicking algorithm for the X-Trans CFA as part of my master thesis. Actually the goal of the thesis was to compare various CFAs which is not so easy because the quality of the resulting image does not only depend on the used CFA but also on the used demosaicking algorithm. But I don´t want to bother you with this here. Instead, I want to share my algorithm. I have written it in Matlab. Below you will find the links to the source files.

XTrans_to_RGB.m
RGB_to_XTrans.m
Test.m

I have to say I´m sorry for the missing comments in the source files. Let me briefly explain how it works. Best is to start with the test.m file. You can use it to load an RGB image and convert it into an X-Trans image (RGB_to_XTrans) or to load a tiff which contains the raw sensor data. The program is not ably to directly read *.RAF files. I used DCRaw with the options '-E -T -4' to convert the *.RAF into a 16bit TIFF.
After loading the image file, it will be converted into a full RGB image. No color correction will be applied to the image, this is why RAW files will appear dark after they have been processed. If you use RGB images for testing, however, then the result should be fine.
Below you'll find some processed test images (top) and the original image (bottom) to compare.




Well, decide for your own, but I think the quality is pretty good and for sure better than what ACDSee can achieve. But it´s not perfect. Let me discuss a few issues:

• The images have a strange pattern on their edges. This is not really a problem of the algorithm. It´s simply because my algorithm is just a prototype and I didn´t care about the edges. Maybe this is also what happens in ACDSee´s processing (but theirs is not a prototype). It could easily be fixed by adding a few pixel rows/columns before demosaicking and crop them afterwards.
• Loss of detail, e.g. in the grass in the first image. Well, of course. I can´t do magic.
• Strange colors on fine details, e.g. on the trousers of the driver in the foreground in the last image or in the water in front of the boat in the second image. The X-Trans CFA has a rather low color resolution. Without pre-filtering the image we should expect to see things like these. Maybe it is possible to fix it by tweaking the algorithm (e.g. increase the number of runs for the alias canceling), however, there´s always a tradeoff. Fixing this issue will most probably result in bleeding colors (like in the 'numbers' image in my last post).

Let me add a few words on how the algorithm works. I basically use the main principles of Hirakawa´s 'Adaptive Homogeneity-Directed Demosaicing Algorithm'. First, I interpolate the green channel in for directions (horizontally, vertically, and 2x diagonally). After that I interpolate the red and blue channels in the same way. For this I exploit the interchannel correlation. The result are four candidate images. In the next step I rate at every pixel location from which candidate image to take the RGB values to construct the final image. After that I add some alias canceling to suppress color seams. That´s it.

Feel free to play around with my algorithm. I will be happy about every feedback. Please don´t use it commercially!