Reconstruct a missing RGB channel

Look at these photographs of natural scenes, with one of the RGB channels taken out of them:

Source (with red): https://en.wikipedia.org/wiki/File:Altja_j%C3%B5gi_Lahemaal.jpg

Source (with green): https://commons.wikimedia.org/wiki/File:2007_mather-lake_hg.jpg

Source (with blue): https://commons.wikimedia.org/wiki/File:Fox_01.jpg

Even without one of the channels, you can probably make out what colours certain things are supposed to be, or at least have a good idea if a reconstruction of the missing channel is accurate.

For example, here is the first picture with a red channel added back onto it which is just random noise:

This image clearly isn't an accurate reconstruction of the red channel. This is because nature photos generally don't use the entire RGB spectrum, but only a subset of "natural-looking colours". Also, the shades of red will follow certain gradients that correlate with the others.

Your task is to build a program that will take a photograph that's had one channel removed and attempt to approximate the original image as closely as possible by reconstructing what it believes is the missing channel.

Your program will be scored on how closely the missing channel matches with the channel of the original image, by counting the percentage of pixels whose values of that channel are within ±15 (inclusive) of the actual value in the original image (where the values range from 0 to 255 as in a standard 8-bit colour channel).

The test cases that your program will be scored on can be found here (9.04 MB zip file, 6 images). Currently it only contains the three example images above and their originals, but I'll add a few more later to make the full suite once I've done them.

Each image is scaled down and cropped to 1024 x 768, so they will have the same weight in your score. The program that can predict the most pixels within the given tolerance wins.

• Could you write a scoring program, e.g. in Python?
– orlp
May 3, 2016 at 7:29
• I'll get to that tomorrow. Right now I need to go to bed. May 3, 2016 at 7:30
• Is there a limit to runtime of the program? May 4, 2016 at 13:43
• @Lause The limit is your patience for the program to actually produce output. May 4, 2016 at 14:21

Python 3 + scikit-image

Simply sets the color of the missing channel to the average of the other two.

import sys
from skimage import io, color

h, w, c = im.shape

removed_channel_options = {0, 1, 2}
for y in range(h):
for x in range(w):
if len(removed_channel_options) == 1: break
removed_channel_options -= {i for i, c in enumerate(im[y][x]) if c > 0}
removed_channel = removed_channel_options.pop()

for y in range(h):
for x in range(w):
p = [float(c) / 255 for c in im[y][x][:3]]
p = [sum(p)/2 if i == removed_channel else p[i]
for i in range(3)]
im[y][x] = [int(c*255) for c in p] + [255]*(c == 4)

io.imsave(sys.argv[2], im)


And the restored images:

• average of the other two - apparently a lot more accurate (subjectively) than I would have guessed. +1. May 3, 2016 at 15:53
• They are reminiscent of hand tinted daguerreotypes of long ago. Very nice. +1
– user15259
May 3, 2016 at 18:20

Lua, Love2D

Just sets the missing channel to the smaller of the two remaining channels.

local inp = love.image.newImageData(arg[2])

local channels = {1, 2, 3}
local removed = nil
local removed_options = {true,true,true}

inp:mapPixel(function(x,y,r,g,b)
local o = {r,g,b}
for k,v in pairs(o) do
if v > 0 then
removed_options[k] = false
end
end
return r,g,b
end)

for k,v in pairs(removed_options) do
if v then
removed = k
break
end
end
inp:mapPixel(function(x,y,r,g,b)
local o = {r,g,b}
o[removed] = math.min(o[removed%3+1], o[(removed+1)%3 + 1])
return unpack(o)
end)

inp:encode('png', IMAGE:gsub("%.png", "2.png"))


Takes a filename as an argument on the command line.

Bonus

I tried setting the missing channel to 255-(a+b), clamping the value. As to "fill in the gap". The results are useless but glorious.

And with 255-(a+b)/2

• The images look even more realistic than the other answer. Great job! Out of curiousity, what happens if you do 255-(a+b)/2 instead (no need for clamping)? Oct 21, 2016 at 1:17
• Results are a less saturated version of the ones posted above. I'll add them to the answer for the giggles. Oct 21, 2016 at 1:29
• I did, in fact, giggle. Thank you. Oct 21, 2016 at 1:50

A little late to the party but I created a ControlNet - Channels RGB machine learning model which was trained on 400K RGB images where each of them have one channel missing in order to restore a missing channel from an image. It uses Stable Diffusion 1.5 generative image AI to infer an image. You can either use the inference as is, or only use the missing channel from the inference (which doesn't look to good unfortunately). I also created an accompanying space which lets you replace the channel from an inference image. Original post is here: https://github.com/lllyasviel/ControlNet/discussions/567 . There are some disadvantages however:

1. it changes the image content a bit,
2. Stable Diffusion 1 works best on images with 512 x n*64.
3. you have to create a prompt for the image, and
4. you have to cherry-pick one of the generations.

Results

first is inference only, second is recomposing the missing channel from inference, the image on the right is ground truth with a missing channel

Forest

Glacier

Fox

• Welcome to Code Golf, and nice answer! Oct 25, 2023 at 16:08