Is it possible to Colorize old photos with Artificial Intelligence or AI?

With new developments in computer power so comes AI. This raises the question Is it possible to Colorize old photos with Artificial Intelligence or AI? Let us delve into what is currently available in 2020 and what to expect in the future.

Updated for 2022!

Basic object colouring.

If you are an artist and want to draw but not colorize, there are some tools available we can use. A website called PaintsChainer uses AI to help color line art. Many artists draw cartoons and line-based artworks. The tedious part for some is colorizing the art. I uploaded a line drawing art piece to their website to see how it works.

With PaintsChainer you can use the brush to help the AI paint in particular colours. You can choose between 3 styles and for an “artistic” interpretation it works quite well.

Using a public domain photo of a teddy bear converted to black and white I tried again. Not such a good result. Let us take a look at what is on offer colorize photos.

AI coloring for photos

Photographs are more complex than line art and require more complex solutions. demos.algorithmia.com offers a solution for colorizing black and white photos. I put it to the test. In order to get a fair comparison, I used a photo I have already coloured and the same image before it was coloured. Below are some results.

You can see in this example the greenery has worked very well. Unfortunately, the software uses just one shade of green and cannot distinguish between different types of foliage. It cannot identify the types of plant and therefore inserts “Green” as a colour. It may have recognized a brick pattern in the background and added “Brown” and has managed to pick the correct colour for the ground. Most surprisingly this is almost the same colour I used and the colour does not spill outside the confines of the walls.

On closer inspection, the colour spill is more evident.

If you used this for colouring a photo there would be so much clean up to do. Time correcting errors and spills would be better suited to using well-known colouring methods.

Below are 4 more examples. Updated for Feb 2020. I am surprised at how well the second example from colourise.sg. The AI engine does a great job. On closer inspection the mouth colour is too low there is a stray blue patch on the second boy and the skin is not fully coloured. The software does not know they have grime all over their faces of course so just colours the skin one shade of skin tone. Amazing job but still plenty of cleaning up to do.

The latest 2020 AI engine by Myheratige seems to do a very similar job to colourise.sg. I tried the same new AI with the formal garden photo.

In the same area as the original 2018 colourise.sg image the new AI has improved and there is less colour bleeding over the foliage and people, there is still a fair way to go yet. Generally much improved. For historically correct colouring I doubt this will ever work. Knowing what flowers should be what colour and determining Ivy from another type of creeping vine or where twigs should be brown, not green, are all big challenges for AI.

For 2022 there is no real change with AI colouring. I ran the same image through MyHeritage and the results are different but no better. The colours are still bleeding over and missing the woman in the bushes altogether.

What other AI colorizing is there?

Google Photos. Google has commented on and showed videos of their “Google Photos” app with an inbuilt feature that will “colorize”  old black and white photos. So far though I have not seen this feature included in any existing version of the app. The videos show it in action (Google – “colourize with Google photos” ) to find all there is to read. We can look forward to trying this out when it is released soon. Much talked about in May 2018 to be released in a few months, but not seen it yet!

 Photoshop

The 2022 version of Photoshop comes with “Neural Filters” one of which is “Colorize”. After applying this filter to our garden scene it is far worse than any of the others. It can’t distinguish between plants and buildings or people and plants and is a total mess. Avoid this one for now, maybe future versions will be better!

photoshop neural filter colouring failure

Other offerings are out there but so far all colours spill out into each other with ill-defined edges. These include vanceAI and Hotpot which turn out really terrible results. Many companies now offer this AI service but all have major flaws which I’ll discuss below.

What is wrong with AI

The trouble with AI colouring is choice. AI does not make choices we humans can make. For example, it appears to detect bricks and makes them red or brown. Depending on what geographical location the building is, the brick varies in colour and type of materials. What if the bricks are damaged or stained or have moss or lichen growing on them. This presents an even more complex situation.

We have already seen how difficult it is to colour foliage. Even more tricky is to pick the correct lighting and to pick the scene apart to understand where the light is coming from, what time of day it might be by shadows and change the tones and colour of light accordingly.

In the future, AI has a long way to get past identifying faces and shapes of bricks. Will it ever be able to tell the species of plants and therefore their correct colours. How can it tell the exact skin tone of a face or colour of a suit jacket or uniform? It is nigh on impossible to detect where one object stops and another starts.

Historically correct colorizing.

Even more difficult is historic colorizing. How can AI research the photo to find out what era, then what location? From here what colour object should be based upon known information and known facts. Ultimately choosing what colour to use.  It can’t make a choice as such because it only has a set of rules to follow. For exampleMyhertiage colours most uniforms blue but British soldiers wore green. It would need to identify the nationality of the person to pick the right uniform colour! Can you tell what nationality someone is by looking at them? Then how can AI possibly do it!

What will the future bring in AI colorization?

Is it possible Colorize old photos with Artificial Intelligence? The answer is yes but there are many caveats. AI has much to learn and much to achieve. If you need your photo colourized check out my colourizing a black and white photo, main page.