How to determine if skin colour on a photo is correct or not?

Hello all,

Often when taking a photo of people, due to a failure during white balance or significant exposure problems, the tones of people faces are somewhat red or too dark or...

Basically what I am trying to do is to determine if skin colour on a detected face is good or not.

Lets make it simple and do this analysis for one race only - i.e. white people.

I will train algorithm (i.e. SVM) with colour features characteristic for bad and good faces. For face detection I will use Viola Jones algorithm, which will give me face bounding box.

What to do next? Maybe to search for a dominant colour inside a face bounding box and to represent it in a LUV color space?

I guess that way I will (ideally) get two clusters in a 2D space, one representing "good" coloured faces and other representing "bad" coloured faces.

Best regards, Matija.

[Jurgen]

If you want to correct for whitebalance or exposure, testing face color seems a bit roundabout? But I think your initial idea is a good one. Although you might need to cluster in 3D space if wrong exposure also affects brightness.

Answer 1

[Jan]

I do not think that there is something like a "white race". Even "white" people have very different skin colors. See:

• http://www.jaffamood.com/img/funny/faces/faces05.jpg
• http://www.simonhoegsberg.com/faces_of_new_york/images/01_faces.jpg
• http://www.straitpinkie.com/wp-content/uploads/2009/10/Gemma-Atkinson-face-980.jpg

Look at: Google Images: "face". There will neither be a safe distinction between "good" and "bad" face colors, not between "white" and "non-white" races.

To get a good white balance, concentrate on a white object. Fortunately the white area of eyes is visible frequently and it is very reliable, because only a few diseases change this color noticably. From a scientific point of view, using white objects for a white balance is much more direct, than to classify the quality of a color based on different colors.

[Matija]

What I want to detect is this red/magenta effect sometimes added due to poor white balance...see example below...

http://fc02.deviantart.net/fs71/f/2012/025/7/4/profile_picture_by_phil93-d4nn2gt.jpg

To concentrate on eyes I need a good and reliable eye detector - which is not available at the moment. There are some detectors, but in general they are poor.

Maybe I can search for white regions inside face bounding box, but this doesnt seems to me like a handy solution.

Thats why I preffer using face detector. As I said earlier, I will train algorithm (i.e. SVM) with colour features characteristic for bad and good faces. New face will be analyzed and then added to one of these groups.

[Jan]

Please post comments in the comment section and not as an answer. Thanks.

I still think, that it will be impossible to solve your problem, because the natural inter-person variance will conceal the variance caused by white-balance and exposure settings of the image acquisition system. A red skin can be real (too high blood pressure, sunburn, red ambient light) or the result of a bad white balance.

But a cause for the difference between our opinions can be the exact definition of:

 the tones of people faces are somewhat red or too dark or...

What problem do you want to solve? Are you looking for pictures, which contain faces with a color inside a certain range, or for pictures, whose colors are near to the real color of the persons faces? While the first problem can be solved by an automatic classification, I have strong doubts that the second problem can be solved, because you do not have information about the real colors. It is like a measurement of the value 171 and the knowledge, that this can be too high or too low - then there is no chance to estimate the quality of the measurement. Of course you can train the classification such that values near to 180 are assumed as "good", but this is a measurement of your preferences only, and the relation to the real face colors is very weak.

[Matija]

Hello,

I want to solve the first problem: I want to select photos containing faces with a color inside a certain range! Lets say that this range is preferred skin color based on my user study...

For example, I want to work for your company and I am trying to find a good profile photo which will be inserted in my CV. Yes, I want to make a good impression with that photo:-)

If I have 100 portrait photos on my computer (50 of them being good and 50 being bad due to poor lightning, exposure etc.), I want to automatically extract good photos from that bunch. Ideally, he will discard my photos where I am having high blood pressure, sunburn etc. together with photos taken under red ambient light, wrong white balance settings etc. I am aware that system is not capable to distinguish this variations.

Is it clear now?

Now we are back on track: I will than train algorithm (i.e. SVM) with color features characteristic for bad and good faces, good meaning preferred skin color. New face will be analyzed and then added to one of these groups.

Answer 2

[Image Analyst]

A paper on this was presented at the last Color Imaging Conference: http://www.imaging.org/IST/store/epub.cfm?abstrid=46635. Abstract, in part, says "....Through the experimental results, the proposed method achieves preferred skin color reproduction for each race....."

[Matija]

Hello,

I tried hard to understand what they were doing - but I couldn't. Maybe it is due to their poor English as can be seen in Abstract or my poor knowledge.

Their work is about skin detection, finding preferred color for each race based on their user study...But I am missing a part how skin color was determined? Which color space they used?

If you do understand, please help.

To consider the case of image including races, race selection is performed by using luminance level and chromaticity in each skin detected area. First, detected skin region is classified by block using labeling algorithm. After that, average luminance level and chromaticity are calculated for each block and the average chromaticity according to luminance level of each race is calculated by using 200 human images in magazine.

[Image Analyst]

If I remember correctly what I think they did was to collect a bunch of images from around the internet and classify their race. Then they asked people of the same race to judge each photo for how pleasing the skin color of that person was. Then they had an ideal color for that race. Then they would take a test image and figure out how to map the colors so that the face in the test picture ended up being the same color as the "ideal/preferred" color.

[Matija]

This part is clear...It is not clear how they sampled skin color?!

Did they use HSV, RGB, LUV? Did they use all three or only two components?

Never mind...

ImageAnalyst, what do you suggest?

I thought of using LUV or YCbCr color space. If I am interested in luminance/lightness I will have 3D space, if not 2D space.

By sampling good/bad colored faces I will (ideally) get two clusters (SVM, LDA or simmilar); a new image, afer being analyzed, will be added to one of these clusters.

Any other proposal?

[Image Analyst]

I don't remember the details of the paper. I'm virtually certain you will not get two clusters. You will get a continuum of data points with no clear dividing line. What are your ideas to handle that?

[Matija]

ImageAnalyst,

what do you think about detecting eyes and sclera; normally, sclera should be white and could be used as a reference point?

[Jurgen]

Since they mention luminance and chromacity they probably used xyY space, i.e. 3D space. Though LAB or LUV seems to make more sense to me.

As for how the colors were sampled, I guess manually---unless they explicitly say otherwise.

But I agree with the others that the hardest part is defining what is right and what is not right, i.e. finding a good training set for your purpose.

[Image Analyst]

You could try that if you want, as long as you have enough pixels there to get a representative sample.

[Image Analyst]

The colors were not sampled manually. If I'm remembering the right paper they used a slight variation of a skin color detection algorithm proposed by someone else.